org.cripac.isee.vpe.util.hdfs

Class HadoopHelper

java.lang.Object

org.cripac.isee.vpe.util.hdfs.HadoopHelper

public class HadoopHelper
extends Object

The HadoopHelper class provides utilities for Hadoop usage.

Author:

Ken Yu, CRIPAC, 2016

Constructor Summary

Constructors

Constructor and Description
HadoopHelper()

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.conf.Configuration	getDefaultConf()
static String	getTrackletInfo(String storeDir) Get the content of info.txt in Har.
static Tracklet	retrieveTracklet(String storeDir) Retrieve a tracklet from the HDFS or HAR.
static Tracklet	retrieveTracklet(String storeDir, org.apache.hadoop.fs.FileSystem hdfs) Retrieve a tracklet from the HDFS or HAR.
static void	storeTracklet(String storeDir, Tracklet tracklet, org.apache.hadoop.fs.FileSystem hdfs) Store a tracklet to the HDFS.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopHelper

public HadoopHelper()

Method Detail

getDefaultConf

public static org.apache.hadoop.conf.Configuration getDefaultConf()

retrieveTracklet

@Nonnull
public static Tracklet retrieveTracklet(@Nonnull
                                                 String storeDir)
                                          throws IOException,
                                                 URISyntaxException

Retrieve a tracklet from the HDFS or HAR. Since a tracklet might be deleted from HDFS during reading, it is highly recommended to retry this function on failure, and the next time it will find the tracklet from HAR.

Parameters:

storeDir - the directory storing the tracklet (including only data of this tracklet).

Returns:

the track retrieved.

Throws:

IOException - on failure of retrieving the tracklet.

URISyntaxException - on syntax error detected in the storeDir.

retrieveTracklet

@Nonnull
public static Tracklet retrieveTracklet(@Nonnull
                                                 String storeDir,
                                                 @Nonnull
                                                 org.apache.hadoop.fs.FileSystem hdfs)
                                          throws IOException,
                                                 URISyntaxException

Retrieve a tracklet from the HDFS or HAR. Since a tracklet might be deleted from HDFS during reading, it is highly recommended to retry this function on failure, and the next time it will find the tracklet from HAR.

Parameters:

storeDir - the directory storing the tracklet (including only data of this tracklet).

Returns:

the track retrieved.

Throws:

IOException - on failure of retrieving the tracklet.

URISyntaxException - on syntax error detected in the storeDir.

getTrackletInfo

public static String getTrackletInfo(@Nonnull
                                     String storeDir)
                              throws IOException,
                                     URISyntaxException

Get the content of info.txt in Har.

Parameters:

storeDir - the directory storing the tracklet.

Returns:

the content in info.txt which is in json format (as a string).

Throws:

IOException - on failure of retrieving the tracklet.

URISyntaxException - on syntax error detected in the storeDir.

storeTracklet

public static void storeTracklet(@Nonnull
                                 String storeDir,
                                 @Nonnull
                                 Tracklet tracklet,
                                 @Nonnull
                                 org.apache.hadoop.fs.FileSystem hdfs)
                          throws Exception

Store a tracklet to the HDFS.

Parameters:

storeDir - the directory storing the tracklet.

tracklet - the tracklet to store.

Throws:

IOException - on failure creating and writing files in HDFS.

Exception