org.cripac.isee.vpe.data

Class DataManagingApp

java.lang.Object

org.cripac.isee.vpe.common.SparkStreamingApp

org.cripac.isee.vpe.data.DataManagingApp

All Implemented Interfaces:

Serializable

public class DataManagingApp
extends SparkStreamingApp

The DataManagingApp class combines two functions: meta data saving and data feeding. The meta data saving function saves meta data, which may be the results of vision algorithms, to HDFS and Neo4j database. The data feeding function retrieves stored results and sendWithLog them to algorithm modules from HDFS and Neo4j database. The reason why combine these two functions is that they should both be modified when and only when a new data inputType shall be supported by the system, and they require less resources than other modules, so combining them can save resources while not harming performance.

Author:

Ken Yu, CRIPAC, 2016

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DataManagingApp.AppPropertyCenter
static class	DataManagingApp.AttrSavingStream
static class	DataManagingApp.IDRankSavingStream
(package private) static class	DataManagingApp.TrackletPackingThread This is a thread independent from Spark Streaming, which listen to tracklet packing jobs from Kafka, and perform HAR packing.
static class	DataManagingApp.TrackletSavingStream
static class	DataManagingApp.VideoCuttingStream

Field Summary

Fields

Modifier and Type	Field and Description
static String	APP_NAME The name of this application.
private static long	serialVersionUID

Fields inherited from class org.cripac.isee.vpe.common.SparkStreamingApp

loggerSingleton

Constructor Summary

Constructors

Constructor and Description
DataManagingApp(DataManagingApp.AppPropertyCenter propCenter)

Method Summary

Modifier and Type	Method and Description
void	addToContext() Add streaming actions directly to the global streaming context.
static void	main(String[] args)

Methods inherited from class org.cripac.isee.vpe.common.SparkStreamingApp

awaitTermination, buildDirectStream, buildDirectStream, finalize, initialize, registerStreams, start, stop

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

APP_NAME

public static final String APP_NAME

The name of this application.

See Also:

Constant Field Values

serialVersionUID

private static final long serialVersionUID

See Also:

Constant Field Values

Constructor Detail

DataManagingApp

public DataManagingApp(DataManagingApp.AppPropertyCenter propCenter)
                throws Exception

Throws:

Exception

Method Detail

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

addToContext

public void addToContext()
                  throws Exception

Description copied from class: SparkStreamingApp

Add streaming actions directly to the global streaming context. This is for applications that may take as input messages with types other than TaskData. Actions that take TaskData as input should be implemented in the Stream.addToGlobalStream(Map), in order to save time of deserialization. Note that existence of Kafka topics used in this method is not automatically checked.

Specified by:

addToContext in class SparkStreamingApp

Throws:

Exception