org.apache.pig.backend.hadoop.executionengine
Class HExecutionEngine

java.lang.Object
  extended by org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
All Implemented Interfaces:
ExecutionEngine

public class HExecutionEngine
extends Object
implements ExecutionEngine


Field Summary
protected  DataStorage ds
           
static String JOB_TRACKER_LOCATION
           
protected  org.apache.hadoop.mapred.JobClient jobClient
           
protected  Map<OperatorKey,OperatorKey> logicalToPhysicalKeys
           
protected  Map<OperatorKey,MapRedResult> materializedResults
           
protected  Map<OperatorKey,ExecPhysicalOperator> physicalOpTable
           
protected  PigContext pigContext
           
 
Constructor Summary
HExecutionEngine(PigContext pigContext)
           
 
Method Summary
 Collection<String> activeScopes()
          List scopes that are active in the back-end
 void close()
          Clean-up and releasing of resources.
 ExecPhysicalPlan compile(ExecLogicalPlan[] plans, Properties properties)
           
 ExecPhysicalPlan compile(ExecLogicalPlan plan, Properties properties)
          Compiles a logical plan into a physical plan, given a set of configuration properties that apply at the plan-level.
 ExecJob execute(ExecPhysicalPlan plan)
          Execute the physical plan in blocking mode.
 Properties getConfiguration()
          Provides configuration information about the execution engine itself.
 DataStorage getDataStorage()
           
 org.apache.hadoop.mapred.JobClient getJobClient()
           
 Map<OperatorKey,MapRedResult> getMaterializedResults()
           
 Map<OperatorKey,ExecPhysicalOperator> getPhysicalOpTable()
           
 Map<String,Object> getStatistics()
          Provides statistics on the Execution Engine: number of nodes, node failure rates, average load, average job wait time...
 void init()
          Place holder for possible initialization activities.
 void init(Properties properties)
           
 void reclaimScope(String scope)
          A mechanism to communicate to the back-end that a set of logical plans go out of scope
 Collection<ExecJob> runningJobs(Properties properties)
          Return currently running jobs (can be useful for admin purposes)
 ExecJob submit(ExecPhysicalPlan plan)
          Execute the physical plan in non-blocking mode
 void updateConfiguration(Properties newConfiguration)
          Provides a way to dynamically change configuration parameters at the Execution Engine level.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

JOB_TRACKER_LOCATION

public static final String JOB_TRACKER_LOCATION
See Also:
Constant Field Values

pigContext

protected PigContext pigContext

ds

protected DataStorage ds

jobClient

protected org.apache.hadoop.mapred.JobClient jobClient

logicalToPhysicalKeys

protected Map<OperatorKey,OperatorKey> logicalToPhysicalKeys

physicalOpTable

protected Map<OperatorKey,ExecPhysicalOperator> physicalOpTable

materializedResults

protected Map<OperatorKey,MapRedResult> materializedResults
Constructor Detail

HExecutionEngine

public HExecutionEngine(PigContext pigContext)
Method Detail

getJobClient

public org.apache.hadoop.mapred.JobClient getJobClient()

getMaterializedResults

public Map<OperatorKey,MapRedResult> getMaterializedResults()

getPhysicalOpTable

public Map<OperatorKey,ExecPhysicalOperator> getPhysicalOpTable()

getDataStorage

public DataStorage getDataStorage()
Specified by:
getDataStorage in interface ExecutionEngine

init

public void init()
          throws ExecException
Description copied from interface: ExecutionEngine
Place holder for possible initialization activities.

Specified by:
init in interface ExecutionEngine
Throws:
ExecException

init

public void init(Properties properties)
          throws ExecException
Throws:
ExecException

close

public void close()
           throws ExecException
Description copied from interface: ExecutionEngine
Clean-up and releasing of resources.

Specified by:
close in interface ExecutionEngine
Throws:
ExecException

getConfiguration

public Properties getConfiguration()
                            throws ExecException
Description copied from interface: ExecutionEngine
Provides configuration information about the execution engine itself.

Specified by:
getConfiguration in interface ExecutionEngine
Returns:
- information about the configuration used to connect to execution engine
Throws:
ExecException

updateConfiguration

public void updateConfiguration(Properties newConfiguration)
                         throws ExecException
Description copied from interface: ExecutionEngine
Provides a way to dynamically change configuration parameters at the Execution Engine level.

Specified by:
updateConfiguration in interface ExecutionEngine
Parameters:
newConfiguration - - the new configuration settings
Throws:
ExecException

getStatistics

public Map<String,Object> getStatistics()
                                 throws ExecException
Description copied from interface: ExecutionEngine
Provides statistics on the Execution Engine: number of nodes, node failure rates, average load, average job wait time...

Specified by:
getStatistics in interface ExecutionEngine
Returns:
ExecutionEngineProperties
Throws:
ExecException

compile

public ExecPhysicalPlan compile(ExecLogicalPlan plan,
                                Properties properties)
                         throws ExecException
Description copied from interface: ExecutionEngine
Compiles a logical plan into a physical plan, given a set of configuration properties that apply at the plan-level. For instance desired degree of parallelism for this plan, which could be different from the "default" one set at the execution engine level.

Specified by:
compile in interface ExecutionEngine
Returns:
physical plan
Throws:
ExecException

compile

public ExecPhysicalPlan compile(ExecLogicalPlan[] plans,
                                Properties properties)
                         throws ExecException
Specified by:
compile in interface ExecutionEngine
Throws:
ExecException

execute

public ExecJob execute(ExecPhysicalPlan plan)
                throws ExecException
Description copied from interface: ExecutionEngine
Execute the physical plan in blocking mode.

Specified by:
execute in interface ExecutionEngine
Throws:
ExecException

submit

public ExecJob submit(ExecPhysicalPlan plan)
               throws ExecException
Description copied from interface: ExecutionEngine
Execute the physical plan in non-blocking mode

Specified by:
submit in interface ExecutionEngine
Throws:
ExecException

runningJobs

public Collection<ExecJob> runningJobs(Properties properties)
                                throws ExecException
Description copied from interface: ExecutionEngine
Return currently running jobs (can be useful for admin purposes)

Specified by:
runningJobs in interface ExecutionEngine
Returns:
Throws:
ExecException

activeScopes

public Collection<String> activeScopes()
                                throws ExecException
Description copied from interface: ExecutionEngine
List scopes that are active in the back-end

Specified by:
activeScopes in interface ExecutionEngine
Returns:
Throws:
ExecException

reclaimScope

public void reclaimScope(String scope)
                  throws ExecException
Description copied from interface: ExecutionEngine
A mechanism to communicate to the back-end that a set of logical plans go out of scope

Specified by:
reclaimScope in interface ExecutionEngine
Throws:
ExecException


Copyright © ${year} The Apache Software Foundation