Building and Running CDAP Applications

In the examples, we refer to the CDAP Sandbox as CDAP, and the example code that is running on it as an application. We'll assume that you are running your application in the default namespace; if not, you will need to adjust commands accordingly. For example, in a URL such as:

http://localhost:11015/v3/namespaces/default/apps...

to use the namespace my_namespace, you would replace default with my_namespace:

http://localhost:11015/v3/namespaces/my_namespace/apps...

Accessing CLI, curl, and the CDAP Sandbox bin

  • For brevity in the commands given below, we will simply use cdap cli for the CDAP Command Line Interface. Substitute the actual path of ./<CDAP-HOME>/bin/cdap cli, or <CDAP-HOME>\bin\cdap.bat cli on Windows, as appropriate.

  • A Windows-version of the application curl is included in the CDAP Sandbox as libexec\bin\curl.exe; use it as a substitute for curl in examples.

  • If you add the CDAP Sandbox bin directory to your path, you can simplify the commands. From within the <CDAP-HOME> directory, enter:

    $ export PATH=${PATH}:`pwd`/bin
    
    > set path=%PATH%;%CD%\bin;%CD%\libexec\bin
    

    The Windows path has been augmented with a directory where the CDAP Sandbox includes Windows-versions of commands such as curl.

Note: There is an issue with running Microsoft Windows and using the CDAP Local Sandbox scripts when CDAP_HOME is defined as a path with spaces in it. Until this is addressed, do not use a path with space characters in it for CDAP_HOME.

Building an Example Application Artifact

From the example's project root (such as examples/<example-dir>), build an example with the Apache Maven command:

$ mvn clean package
> mvn clean package

To build without running tests, use:

$ mvn clean package -DskipTests
> mvn clean package -DskipTests

To build all the examples, switch to the main examples directory and run the Maven command:

$ cd <CDAP-HOME>/examples
$ mvn clean package -DskipTests
> cd <CDAP-HOME>\examples
> mvn clean package -DskipTests

Starting CDAP

Before running an example application, check that an instance of CDAP is running and available; if not, follow the instructions for Starting and Stopping CDAP Sandbox.

If you can reach the CDAP UI through a browser at http://localhost:11011/, CDAP is running.

Deploying an Application

Once CDAP is started, there are multiple ways to deploy an application using an example JAR:

  • Using the green "plus" button to upload the application's JAR file on the CDAP UI

  • Using the Command Line Interface (CLI):

    $ cdap cli load artifact examples/<example-directory>/target/<example>-5.1.2.jar
    Successfully added artifact with name '<example>'
    
    $ cdap cli create app <app name> <example> 5.1.2 user
    Successfully created application
    
    > cdap cli load artifact examples\<example-directory>\target\<example>-5.1.2.jar
    Successfully added artifact with name '<example>'
    
    > cdap cli create app <app name> <example> 5.1.2 user
    Successfully created application
    

    The CLI can be accessed under Windows using the bin\cdap.bat cli script.

  • Using an application such as curl (a Windows-version is included in the CDAP Sandbox in libexec\bin\curl.exe):

    $ curl -w"\n" localhost:11015/v3/namespaces/default/artifacts/<example> \
      --data-binary @examples/<example-directory>/target/<example>-5.1.2.jar
    Artifact added successfully
    
    $ curl -w"\n" -X PUT -H "Content-Type: application/json" localhost:11015/v3/namespaces/default/apps/<app name> \
      -d '{ "artifact": { "name": "<example>", "version": "5.1.2", "scope": "user" }, "config": {} }'
    Deploy Complete
    
    > curl localhost:11015/v3/namespaces/default/artifacts/<example> ^
      --data-binary @examples\<example-directory>\target\<example>-5.1.2.jar
    Artifact added successfully
    
    > curl -X PUT -H "Content-Type: application/json" localhost:11015/v3/namespaces/default/apps/<app name> ^
      -d "{ \"artifact\": { \"name\": \"<example>\", \"version\": \"5.1.2\", \"scope\": \"user\" }, \"config\": {} }"
    Deploy Complete
    

Starting an Application's Programs

Once an application is deployed, there are multiple methods for starting an application's programs:

  • The CDAP UI.

  • The Command Line Interface to start a specific program of an application. (In each CDAP example, the CLI commands for that particular example are provided):

    $ cdap cli start <program-type> <app-id.program-id>
    
    > cdap cli start <program-type> <app-id.program-id>
    
    Parameter Description
    <program-type> One of flow, mapreduce, service, spark, worker, or workflow
    <app-id> Name of the application being called
    <program-id> Name of the flow, MapReduce, service, spark, worker or workflow being called
  • The Command Line Interface to start all or selected types of programs of an application at once:

    $ start app <app-id> programs [of type <program-types>]
    
    > start app <app-id> programs [of type <program-types>]
    
    Parameter Description
    <app-id> Name of the application being called
    <program-types> An optional comma-separated list of program types (flow, mapreduce, service, spark, worker, or workflow) which will start all programs of those types; for example, specifying 'flow,workflow' will start all flows and workflows in the application
  • The Program Lifecycle of the Lifecycle HTTP RESTful API to start the programs of an application using a program such as curl

Stopping an Application's Programs

Once an application is deployed, there are multiple ways for stopping an application's programs:

  • The CDAP UI.

  • The Command Line Interface to stop a specific program of an application:

    $ cdap cli stop <program-type> <app-id.program-id>
    
    > cdap cli stop <program-type> <app-id.program-id>
    
    Parameter Description
    <program-type> One of flow, mapreduce, service, spark, worker, or workflow
    <app-id> Name of the application being called
    <program-id> Name of the flow, MapReduce, service, spark, worker or workflow being called
  • The Command Line Interface to stop all or selected types of programs of an application at once:

    $ stop app <app-id> programs [of type <program-types>]
    
    > stop app <app-id> programs [of type <program-types>]
    
    Parameter Description
    <app-id> Name of the application being called
    <program-types> An optional comma-separated list of program types (flow, mapreduce, service, spark, worker, or workflow) which will stop all programs of those types; for example, specifying 'flow,workflow' will stop all flows and workflows in the application
  • Use the Program Lifecycle of the Lifecycle HTTP RESTful API to stop the programs of an application using a program such as curl

Removing an Application

Once an application is "stopped"—when all of its programs are stopped—you can remove the application using the CDAP UI.

Alternatively, you can also use the Command Line Interface:

$ cdap cli delete app <app-id>
> cdap cli delete app <app-id>

Note that any datasets created or used by the application will remain, as they are independent of the application. Datasets can be deleted from the CDAP UI, or by using the HTTP Restful API, the Java Client API, or the Command Line Interface API.

Streams can be either truncated or deleted, using similar methods.

The artifact used to create the application will also remain, as multiple applications can be created from the same artifact. Artifacts can be deleted using the UI, Http Restful API, the Java Client API, or the Command Line Interface API.