SystemPropertyCenter

java.lang.Object
- org.cripac.isee.vpe.ctrl.SystemPropertyCenter

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

DataManagingApp.AppPropertyCenter, PedestrianAttrRecogApp.AppPropertyCenter, PedestrianTrackingApp.AppPropertyCenter
```
public class SystemPropertyCenter
extends Object
implements Serializable
```
The SystemPropertyCenter class is responsible of managing the properties of the systems. There are some properties predefined, and they can be overwritten by command options or an extern property file. It can also generate back command options for uses like SparkSubmit.

Author:

Ken Yu, CRIPAC, 2016

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`private String`	`appPropFilePath`
`(package private) String[]`	`appsToStart`
`int`	`batchDuration`
`int`	`bufDuration`
`String`	`caffeGPU`
`String`	`checkpointRootDir`
`private int`	`driverCores`
`private String`	`driverMem`
`private int`	`executorCores`
`String`	`executorMem`
`private String`	`hadoopQueue`
`private String`	`hdfsDefaultName`
`private String`	`jarPath`
`String`	`kafkaBootstrapServers`
`private int`	`kafkaFetchTimeoutMs`
`String`	`kafkaLocationStrategy`
`private int`	`kafkaMsgMaxBytes`
`int`	`kafkaNumPartitions`
`int`	`kafkaReplFactor`
`private int`	`kafkaRequestTimeoutMs`
`int`	`kafkaSendMaxSize`
`private String`	`log4jPropFilePath`
`protected Logger`	`logger`
`String`	`maxRatePerPartition`
`String`	`metadataDir`
`int`	`numExecutors`
`int`	`repartition`
`private static long`	`serialVersionUID`
`private String`	`sparkConfFilePath`
`String`	`sparkDeployMode`
`String`	`sparkMaster`
`private String`	`sysPropFilePath`
`protected Properties`	`sysProps`
`boolean`	`taskControllerEnable`
`private int`	`totalExecutorCores`
`boolean`	`verbose`
`private String`	`yarnAmNodeLabelExpression`
`String`	`zkConn`
`int`	`zkConnectionTimeoutMS`
`int`	`zkSessionTimeoutMs`

Constructor Summary

Constructors
Constructor and Description

SystemPropertyCenter(String... args)

Constructors
Constructor and Description
`SystemPropertyCenter(String... args)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`private String[]`	`getArgs(String appName)` Generate command line options for SparkSubmit client, according to the stored properties.
`Properties`	`getKafkaConsumerProp(String group, boolean isStringValue)`
`Map<String,Object>`	`getKafkaParams(String group)`
`Properties`	`getKafkaProducerProp(boolean isStringValue)`
`(package private) org.apache.spark.launcher.SparkLauncher`	`GetSparkLauncher(String appName)`
`private void`	`validateConfigurations()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

serialVersionUID
```
private static final long serialVersionUID
```
See Also:

Constant Field Values

logger
```
protected transient Logger logger
```

zkConn
```
public String zkConn
```

zkSessionTimeoutMs
```
public int zkSessionTimeoutMs
```

zkConnectionTimeoutMS
```
public int zkConnectionTimeoutMS
```

kafkaBootstrapServers
```
public String kafkaBootstrapServers
```

kafkaNumPartitions
```
public int kafkaNumPartitions
```

kafkaReplFactor
```
public int kafkaReplFactor
```

kafkaMsgMaxBytes
```
private int kafkaMsgMaxBytes
```

kafkaSendMaxSize
```
public int kafkaSendMaxSize
```

kafkaRequestTimeoutMs
```
private int kafkaRequestTimeoutMs
```

kafkaFetchTimeoutMs
```
private int kafkaFetchTimeoutMs
```

kafkaLocationStrategy
```
public String kafkaLocationStrategy
```

maxRatePerPartition

@Nullable
public String maxRatePerPartition

checkpointRootDir
```
public String checkpointRootDir
```

metadataDir
```
public String metadataDir
```

sparkMaster
```
public String sparkMaster
```

sparkDeployMode
```
public String sparkDeployMode
```

appsToStart
```
String[] appsToStart
```

caffeGPU
```
public String caffeGPU
```

numExecutors
```
public int numExecutors
```

executorMem
```
public String executorMem
```

executorCores
```
private int executorCores
```

totalExecutorCores
```
private int totalExecutorCores
```

driverMem
```
private String driverMem
```

driverCores
```
private int driverCores
```

yarnAmNodeLabelExpression

private String yarnAmNodeLabelExpression

hadoopQueue
```
private String hadoopQueue
```

sysPropFilePath
```
private String sysPropFilePath
```

appPropFilePath
```
private String appPropFilePath
```

sparkConfFilePath
```
private String sparkConfFilePath
```

log4jPropFilePath
```
private String log4jPropFilePath
```

hdfsDefaultName
```
private String hdfsDefaultName
```

jarPath
```
private String jarPath
```

bufDuration
```
public int bufDuration
```

batchDuration
```
public int batchDuration
```

repartition
```
public int repartition
```

verbose
```
public boolean verbose
```

taskControllerEnable
```
public boolean taskControllerEnable
```

sysProps
```
protected Properties sysProps
```

Constructor Detail

SystemPropertyCenter

public SystemPropertyCenter(@Nonnull
                            String... args)
                     throws URISyntaxException,
                            ParserConfigurationException,
                            SAXException

Throws:: URISyntaxException; ParserConfigurationException; SAXException

Method Detail

validateConfigurations
```
private void validateConfigurations()
```

getArgs
```
private String[] getArgs(String appName)
```
Generate command line options for SparkSubmit client, according to the stored properties.

Returns:

An array of string with format required by SparkSubmit client.

GetSparkLauncher

org.apache.spark.launcher.SparkLauncher GetSparkLauncher(String appName)
                                                  throws IOException

Throws:: IOException

getKafkaProducerProp

public Properties getKafkaProducerProp(boolean isStringValue)

getKafkaConsumerProp

public Properties getKafkaConsumerProp(String group,
                                       boolean isStringValue)

getKafkaParams

public Map<String,Object> getKafkaParams(String group)

Class SystemPropertyCenter

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

serialVersionUID

logger

zkConn

zkSessionTimeoutMs

zkConnectionTimeoutMS

kafkaBootstrapServers

kafkaNumPartitions

kafkaReplFactor

kafkaMsgMaxBytes

kafkaSendMaxSize

kafkaRequestTimeoutMs

kafkaFetchTimeoutMs

kafkaLocationStrategy

maxRatePerPartition

checkpointRootDir

metadataDir

sparkMaster

sparkDeployMode

appsToStart

caffeGPU

numExecutors

executorMem

executorCores

totalExecutorCores

driverMem

driverCores

yarnAmNodeLabelExpression

hadoopQueue

sysPropFilePath

appPropFilePath

sparkConfFilePath

log4jPropFilePath

hdfsDefaultName

jarPath

bufDuration

batchDuration

repartition

verbose

taskControllerEnable

sysProps

Constructor Detail

SystemPropertyCenter

Method Detail

validateConfigurations

getArgs

GetSparkLauncher

getKafkaProducerProp

getKafkaConsumerProp

getKafkaParams