|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.pig.backend.executionengine.PigSlicer
public class PigSlicer
Creates a slice per block size element in all files at location. If location is a glob or a directory, slices are created for every matched file.
If individual files at location end with .gz
or
.bz2
, they will be decompressed before being passed on to the
LoadFunc.
Constructor Summary | |
---|---|
PigSlicer(String funcSpec)
|
Method Summary | |
---|---|
void |
setSplittable(boolean splittable)
|
Slice[] |
slice(DataStorage store,
String location)
Creates slices of data from store at location . |
void |
validate(DataStorage store,
String location)
Checks that location is parsable by this Slicer, and that
if the DataStorage is used by the Slicer, it's readable from there. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public PigSlicer(String funcSpec)
funcSpec
- -
the funcSpec for a LoadFunc that can process the data at
location.Method Detail |
---|
public void setSplittable(boolean splittable)
public Slice[] slice(DataStorage store, String location) throws IOException
Slicer
store
at location
.
slice
in interface Slicer
IOException
public void validate(DataStorage store, String location) throws IOException
Slicer
location
is parsable by this Slicer, and that
if the DataStorage is used by the Slicer, it's readable from there. If it
isn't, an IOException with a message explaining why will be thrown.
This does not ensure that all the data in location
is
valid. It's a preflight check that there's some chance of the Slicer
working before actual Slices are created and sent off for processing.
validate
in interface Slicer
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |