org.apache.hadoop.dfs
Class DataNode

java.lang.Object
  extended by org.apache.hadoop.dfs.DataNode
All Implemented Interfaces:
Runnable, FSConstants

public class DataNode
extends Object
implements FSConstants, Runnable

DataNode is a class (and program) that stores a set of blocks for a DFS deployment. A single deployment can have one or many DataNodes. Each DataNode communicates regularly with a single NameNode. It also communicates with client code and other DataNodes from time to time. DataNodes store a series of named blocks. The DataNode allows client code to read these blocks, or to write new block data. The DataNode may also, in response to instructions from its NameNode, delete blocks or copy blocks to/from other DataNodes. The DataNode maintains just one critical table: block-> stream of bytes (of BLOCK_SIZE or less) This info is stored on a local disk. The DataNode reports the table's contents to the NameNode upon startup and every so often afterwards. DataNodes spend their lives in an endless loop of asking the NameNode for something to do. A NameNode cannot connect to a DataNode directly; a NameNode simply returns values from functions invoked by a DataNode. DataNodes maintain an open server socket so that client code or other DataNodes can read/write data. The host/port for this server is reported to the NameNode, which then sends that information to clients or other DataNodes that might be interested.


Nested Class Summary
 
Nested classes/interfaces inherited from interface org.apache.hadoop.dfs.FSConstants
FSConstants.CheckpointStates, FSConstants.DatanodeReportType, FSConstants.NodeType, FSConstants.SafeModeAction, FSConstants.StartupOption, FSConstants.UpgradeAction
 
Field Summary
static org.apache.commons.logging.Log LOG
           
 
Fields inherited from interface org.apache.hadoop.dfs.FSConstants
BLOCK_INVALIDATE_CHUNK, BLOCKREPORT_INITIAL_DELAY, BLOCKREPORT_INTERVAL, BUFFER_SIZE, COMPLETE_SUCCESS, DATA_TRANSFER_VERSION, DEFAULT_BLOCK_SIZE, DEFAULT_DATA_SOCKET_SIZE, HEARTBEAT_INTERVAL, LAYOUT_VERSION, LEASE_HARDLIMIT_PERIOD, LEASE_SOFTLIMIT_PERIOD, MAX_PATH_DEPTH, MAX_PATH_LENGTH, MIN_BLOCKS_FOR_WRITE, OP_ACK, OP_BLOCKRECEIVED, OP_BLOCKREPORT, OP_CLIENT_ABANDONBLOCK, OP_CLIENT_ABANDONBLOCK_ACK, OP_CLIENT_ADDBLOCK, OP_CLIENT_ADDBLOCK_ACK, OP_CLIENT_COMPLETEFILE, OP_CLIENT_COMPLETEFILE_ACK, OP_CLIENT_DATANODE_HINTS, OP_CLIENT_DATANODE_HINTS_ACK, OP_CLIENT_DATANODEREPORT, OP_CLIENT_DATANODEREPORT_ACK, OP_CLIENT_DELETE, OP_CLIENT_DELETE_ACK, OP_CLIENT_EXISTS, OP_CLIENT_EXISTS_ACK, OP_CLIENT_ISDIR, OP_CLIENT_ISDIR_ACK, OP_CLIENT_LISTING, OP_CLIENT_LISTING_ACK, OP_CLIENT_MKDIRS, OP_CLIENT_MKDIRS_ACK, OP_CLIENT_OBTAINLOCK, OP_CLIENT_OBTAINLOCK_ACK, OP_CLIENT_OPEN, OP_CLIENT_OPEN_ACK, OP_CLIENT_RAWSTATS, OP_CLIENT_RAWSTATS_ACK, OP_CLIENT_RELEASELOCK, OP_CLIENT_RELEASELOCK_ACK, OP_CLIENT_RENAMETO, OP_CLIENT_RENAMETO_ACK, OP_CLIENT_RENEW_LEASE, OP_CLIENT_RENEW_LEASE_ACK, OP_CLIENT_STARTFILE, OP_CLIENT_STARTFILE_ACK, OP_CLIENT_TRYAGAIN, OP_COPY_BLOCK, OP_ERROR, OP_FAILURE, OP_HEARTBEAT, OP_INVALIDATE_BLOCKS, OP_READ_BLOCK, OP_READ_METADATA, OP_REPLACE_BLOCK, OP_STATUS_CHECKSUM_OK, OP_STATUS_ERROR, OP_STATUS_ERROR_CHECKSUM, OP_STATUS_ERROR_EXISTS, OP_STATUS_ERROR_INVALID, OP_STATUS_SUCCESS, OP_TRANSFERBLOCKS, OP_TRANSFERDATA, OP_WRITE_BLOCK, OPERATION_FAILED, READ_TIMEOUT, SIZE_OF_INTEGER, SMALL_BUFFER_SIZE, STILL_WAITING, WRITE_TIMEOUT, WRITE_TIMEOUT_EXTENSION
 
Method Summary
static InetSocketAddress createSocketAddr(String target)
          Deprecated. 
static DataNode getDataNode()
          Return the DataNode object
 FSDatasetInterface getFSDataset()
          This method is used for testing.
 String getNamenode()
          Return the namenode's identifier
 InetSocketAddress getNameNodeAddr()
           
 InetSocketAddress getSelfAddr()
           
static void main(String[] args)
           
 void offerService()
          Main loop for the DataNode.
 void run()
          No matter what kind of exception we get, keep retrying to offerService().
 void scheduleBlockReport(long delay)
          This methods arranges for the data node to send the block report at the next heartbeat.
 void shutdown()
          Shut down this instance of the datanode.
 String toString()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG
Method Detail

createSocketAddr

@Deprecated
public static InetSocketAddress createSocketAddr(String target)
                                          throws IOException
Deprecated. 

Use NetUtils.createSocketAddr(String) instead.

Throws:
IOException

getDataNode

public static DataNode getDataNode()
Return the DataNode object


getNameNodeAddr

public InetSocketAddress getNameNodeAddr()

getSelfAddr

public InetSocketAddress getSelfAddr()

getNamenode

public String getNamenode()
Return the namenode's identifier


shutdown

public void shutdown()
Shut down this instance of the datanode. Returns only after shutdown is complete.


offerService

public void offerService()
                  throws Exception
Main loop for the DataNode. Runs until shutdown, forever calling remote NameNode functions.

Throws:
Exception

run

public void run()
No matter what kind of exception we get, keep retrying to offerService(). That's the loop that connects to the NameNode and provides basic DataNode functionality. Only stop when "shouldRun" is turned off (which can only happen at shutdown).

Specified by:
run in interface Runnable

toString

public String toString()
Overrides:
toString in class Object

scheduleBlockReport

public void scheduleBlockReport(long delay)
This methods arranges for the data node to send the block report at the next heartbeat.


getFSDataset

public FSDatasetInterface getFSDataset()
This method is used for testing. Examples are adding and deleting blocks directly. The most common usage will be when the data node's storage is similated.

Returns:
the fsdataset that stores the blocks

main

public static void main(String[] args)


Copyright © 2006 The Apache Software Foundation