org.apache.pig.backend.executionengine
Class PigSlicer

java.lang.Object
  extended by org.apache.pig.backend.executionengine.PigSlicer
All Implemented Interfaces:
Slicer

public class PigSlicer
extends Object
implements Slicer

Creates a slice per block size element in all files at location. If location is a glob or a directory, slices are created for every matched file.

If individual files at location end with .gz or .bz2, they will be decompressed before being passed on to the LoadFunc.


Constructor Summary
PigSlicer(String funcSpec)
           
 
Method Summary
 void setSplittable(boolean splittable)
           
 Slice[] slice(DataStorage store, String location)
          Creates slices of data from store at location.
 void validate(DataStorage store, String location)
          Checks that location is parsable by this Slicer, and that if the DataStorage is used by the Slicer, it's readable from there.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

PigSlicer

public PigSlicer(String funcSpec)
Parameters:
funcSpec - - the funcSpec for a LoadFunc that can process the data at location.
Method Detail

setSplittable

public void setSplittable(boolean splittable)

slice

public Slice[] slice(DataStorage store,
                     String location)
              throws IOException
Description copied from interface: Slicer
Creates slices of data from store at location.

Specified by:
slice in interface Slicer
Returns:
the Slices to be serialized and sent out to nodes for processing.
Throws:
IOException

validate

public void validate(DataStorage store,
                     String location)
              throws IOException
Description copied from interface: Slicer
Checks that location is parsable by this Slicer, and that if the DataStorage is used by the Slicer, it's readable from there. If it isn't, an IOException with a message explaining why will be thrown.

This does not ensure that all the data in location is valid. It's a preflight check that there's some chance of the Slicer working before actual Slices are created and sent off for processing.

Specified by:
validate in interface Slicer
Throws:
IOException


Copyright © ${year} The Apache Software Foundation