๐Ÿ”—Hello World

The simplest Cask Data Application Platform (CDAP) example.

๐Ÿ”—Overview

This application uses one stream, one dataset, one flow, and one service to implement the classic "Hello World":

  • A stream to send names to;
  • A flow with a single flowlet that reads the stream and stores in a dataset each name in a KeyValueTable; and
  • A service that reads the name from the KeyValueTable and responds with "Hello [Name]!"

๐Ÿ”—The HelloWorld Application

public class HelloWorld extends AbstractApplication {

  @Override
  public void configure() {
    setName("HelloWorld");
    setDescription("A Hello World program for the Cask Data Application Platform");
    addStream(new Stream("who"));
    createDataset("whom", KeyValueTable.class, DatasetProperties.builder().setDescription("Store names").build());
    addFlow(new WhoFlow());
    addService(new Greeting());
  }
...

The application uses a stream called who to ingest data through a flow WhoFlow to a dataset whom.

๐Ÿ”—The WhoFlow

This is a trivial flow with a single flowlet named saver of type NameSaver:

public static final class WhoFlow extends AbstractFlow {

  @Override
  protected void configure() {
    setName("WhoFlow");
    setDescription("A flow that collects names");
    addFlowlet("saver", new NameSaver());
    connectStream("who", "saver");
  }
}

The flowlet uses a dataset of type KeyValueTable to store the names it reads from the stream. Every time a new name is received, it is stored in the table under the key name, and it overwrites any name that was previously stored:

public static final class NameSaver extends AbstractFlowlet {

  static final byte[] NAME = { 'n', 'a', 'm', 'e' };

  @UseDataSet("whom")
  private KeyValueTable whom;

  private Metrics metrics;

  @ProcessInput
  public void process(StreamEvent event) {
    byte[] name = Bytes.toBytes(event.getBody());
    if (name.length > 0) {
      whom.write(NAME, name);

      if (name.length > 10) {
        metrics.count("names.longnames", 1);
      }
      metrics.count("names.bytes", name.length);
    }
  }
}

Note that the flowlet also emits metrics: every time a name longer than 10 characters is received, the counter names.longnames is incremented by one, and the metric names.bytes is incremented by the length of the name. We will see below how to retrieve these metrics using the Metrics HTTP RESTful API.

๐Ÿ”—The Greeting Service

This is a simple service. It has only one handler, the class GreetingHandler.

public static final class Greeting extends AbstractService {

  public static final String SERVICE_NAME = "Greeting";

  @Override
  protected void configure() {
    setName(SERVICE_NAME);
    setDescription("Service that creates a greeting using a user's name.");
    addHandler(new GreetingHandler());
  }
}

๐Ÿ”—The GreetingHandler Handler

This has a single endpoint called greet that does not accept arguments. When invoked, it reads the name stored by the NameSaver from the key-value table. It returns a simple greeting containing that name:

public static final class GreetingHandler extends AbstractHttpServiceHandler {

  @UseDataSet("whom")
  private KeyValueTable whom;

  private Metrics metrics;

  @Path("greet")
  @GET
  public void greet(HttpServiceRequest request, HttpServiceResponder responder) {
    byte[] name = whom.read(NameSaver.NAME);
    String toGreet = name != null ? new String(name, Charsets.UTF_8) : "World";
    if (toGreet.equals("Jane Doe")) {
      metrics.count("greetings.count.jane_doe", 1);
    }
    responder.sendString(String.format("Hello %s!", toGreet));
  }
}

Note that the service, like the flowlet, also emits metrics: every time the name Jane Doe is received, the counter greetings.count.jane_doe is incremented by one. We will see below how to retrieve this metric using the Metrics HTTP RESTful API.

๐Ÿ”—Building and Starting

  • You can build the example as described in Building an Example Application

  • Start CDAP (as described in Starting and Stopping CDAP).

  • Deploy the application, as described in Deploying an Application. For example, from the Standalone CDAP SDK directory, use the Command Line Interface (CLI):

    $ cdap cli load artifact examples/HelloWorld/target/HelloWorld-4.1.1.jar
    
    Successfully added artifact with name 'HelloWorld'
    
    $ cdap cli create app HelloWorld HelloWorld 4.1.1 user
    
    Successfully created application
    
    > cdap cli load artifact examples\HelloWorld\target\HelloWorld-4.1.1.jar
    
    Successfully added artifact with name 'HelloWorld'
    
    > cdap cli create app HelloWorld HelloWorld 4.1.1 user
    
    Successfully created application
    
  • Once the application has been deployed, you can start its components, as described in Starting an Application, and detailed at the start of running the example.

  • Once all components are started, run the example.

  • When finished, you can stop and remove the application.

๐Ÿ”—Running the Example

๐Ÿ”—Starting the Flow

  • Using the CDAP UI, go to the HelloWorld application overview page, programs tab, click WhoFlow to get to the flow detail page, then click the Start button; or

  • From the Standalone CDAP SDK directory, use the Command Line Interface:

    $ cdap cli start flow HelloWorld.WhoFlow
    
    Successfully started flow 'WhoFlow' of application 'HelloWorld' with stored runtime arguments '{}'
    
    > cdap cli start flow HelloWorld.WhoFlow
    
    Successfully started flow 'WhoFlow' of application 'HelloWorld' with stored runtime arguments '{}'
    

๐Ÿ”—Starting the Service

  • Using the CDAP UI, go to the HelloWorld application overview page, programs tab, click Greeting to get to the service detail page, then click the Start button; or

  • From the Standalone CDAP SDK directory, use the Command Line Interface:

    $ cdap cli start service HelloWorld.Greeting
    
    Successfully started service 'Greeting' of application 'HelloWorld' with stored runtime arguments '{}'
    
    > cdap cli start service HelloWorld.Greeting
    
    Successfully started service 'Greeting' of application 'HelloWorld' with stored runtime arguments '{}'
    

๐Ÿ”—Injecting a Name

In the application overview page, programs tab, click on WhoFlow. This takes you to the flow details page. (If you haven't already started the flow, click on the Start button in the right-side, below the green arrow.) The flow's status will read Running when it is ready to receive events.

Now double-click on the who stream on the left side of the flow visualization, which brings up a pop-up window. Enter a name and click the Inject button. After you close the pop-up window, you will see that the counters for both the stream and the saver flowlet increase to 1. You can repeat this step to enter more names, but remember that only the last name is stored in the key-value table.

Metrics are collected based on the bytes metric (the total number of bytes of names), the longnames metric (the number of names, each greater than 10 characters), and the greetings.count.jane_doe metric (the number of times the name Jane Doe has been "greeted").

To try out these metrics, first send a few long names (each greater than 10 characters) and send Jane Doe a number of times.

You can also use the CDAP CLI:

$ cdap cli send stream who "'Alice Cumberbund'"
$ cdap cli send stream who "Bob"
$ cdap cli send stream who "'Jane Doe'"
$ cdap cli send stream who "Tom"
...
> cdap cli send stream who "'Alice Cumberbund'"
> cdap cli send stream who "Bob"
> cdap cli send stream who "'Jane Doe'"
> cdap cli send stream who "Tom"
...

๐Ÿ”—Using the Service

Go back to the application overview page, programs tab, and click on the Greeting service. (If you haven't already started the service, click on the Start button on the right-side.) The service's label will read Running when it is ready to receive events.

Now you can make a request to the service using curl:

$ curl -w"\n" -X GET "http://localhost:11015/v3/namespaces/default/apps/HelloWorld/services/Greeting/methods/greet"
> curl -X GET "http://localhost:11015/v3/namespaces/default/apps/HelloWorld/services/Greeting/methods/greet"

If the last name you entered was Tom, the service will respond with Hello Tom!

There is a Make Request button in the CDAP UI, Greeting service that will make the same request, with a similar response.

๐Ÿ”—Retrieving Metrics

You can now query the metrics that are emitted by the flow and service. The results you receive will vary depending on the entries you have made to the flow. If a particular metric has no value, it will return an empty array in the "series" of the results, such as:

{"startTime":0,"endTime":1429475995,"series":[]}

To see the value of the names.bytes metric, you can make an HTTP request to the Metrics HTTP RESTful API using curl:

$ curl -w"\n" -X POST "http://localhost:11015/v3/metrics/query?tag=namespace:default&tag=app:HelloWorld&tag=flow:WhoFlow&tag=flowlet:saver&metric=user.names.bytes&aggregate=true"

{"startTime":0,"endTime":1458877439,"series":[{"metricName":"user.names.bytes","grouping":{},"data":[{"time":0,"value":79}]}],"resolution":"2147483647s"}
> curl -X POST "http://localhost:11015/v3/metrics/query?tag=namespace:default&tag=app:HelloWorld&tag=flow:WhoFlow&tag=flowlet:saver&metric=user.names.bytes&aggregate=true"

{"startTime":0,"endTime":1458877439,"series":[{"metricName":"user.names.bytes","grouping":{},"data":[{"time":0,"value":79}]}],"resolution":"2147483647s"}

To see the value of the names.longnames metric (the number of names, each of which is greater than 10 characters in length), you can use:

$ curl -w"\n" -X POST "http://localhost:11015/v3/metrics/query?tag=namespace:default&tag=app:HelloWorld&tag=flow:WhoFlow&tag=flowlet:saver&metric=user.names.longnames&aggregate=true"

{"startTime":0,"endTime":1458877544,"series":[{"metricName":"user.names.longnames","grouping":{},"data":[{"time":0,"value":3}]}],"resolution":"2147483647s"}
> curl -X POST "http://localhost:11015/v3/metrics/query?tag=namespace:default&tag=app:HelloWorld&tag=flow:WhoFlow&tag=flowlet:saver&metric=user.names.longnames&aggregate=true"

{"startTime":0,"endTime":1458877544,"series":[{"metricName":"user.names.longnames","grouping":{},"data":[{"time":0,"value":3}]}],"resolution":"2147483647s"}

To see the value of the greetings.count.jane_doe metric (the number of times the specific name Jane Doe has been "greeted"), you can use:

$ curl -w"\n" -X POST "http://localhost:11015/v3/metrics/query?tag=namespace:default&tag=app:HelloWorld&tag=service:Greeting&metric=user.greetings.count.jane_doe&aggregate=true"

{"startTime":0,"endTime":1458877575,"series":[{"metricName":"user.greetings.count.jane_doe","grouping":{},"data":[{"time":0,"value":2}]}],"resolution":"2147483647s"}
> curl -X POST "http://localhost:11015/v3/metrics/query?tag=namespace:default&tag=app:HelloWorld&tag=service:Greeting&metric=user.greetings.count.jane_doe&aggregate=true"

{"startTime":0,"endTime":1458877575,"series":[{"metricName":"user.greetings.count.jane_doe","grouping":{},"data":[{"time":0,"value":2}]}],"resolution":"2147483647s"}

๐Ÿ”—Stopping and Removing the Application

Once done, you can stop the applicationโ€”if it hasn't stopped alreadyโ€”as described in Stopping an Application. Here is an example-specific description of the steps:

Stopping the Flow

  • Using the CDAP UI, go to the HelloWorld application overview page, programs tab, click WhoFlow to get to the flow detail page, then click the Stop button; or

  • From the Standalone CDAP SDK directory, use the Command Line Interface:

    $ cdap cli stop flow HelloWorld.WhoFlow
    
    Successfully stopped flow 'WhoFlow' of application 'HelloWorld'
    
    > cdap cli stop flow HelloWorld.WhoFlow
    
    Successfully stopped flow 'WhoFlow' of application 'HelloWorld'
    

Stopping the Service

  • Using the CDAP UI, go to the HelloWorld application overview page, programs tab, click Greeting to get to the service detail page, then click the Stop button; or

  • From the Standalone CDAP SDK directory, use the Command Line Interface:

    $ cdap cli stop service HelloWorld.Greeting
    
    Successfully stopped service 'Greeting' of application 'HelloWorld'
    
    > cdap cli stop service HelloWorld.Greeting
    
    Successfully stopped service 'Greeting' of application 'HelloWorld'
    

Removing the Application

You can now remove the application as described in Removing an Application, or:

  • Using the CDAP UI, go to the HelloWorld application overview page, programs tab, click the Actions menu on the right side and select Manage to go to the Management pane for the application, then click the Actions menu on the right side and select Delete to delete the application; or

  • From the Standalone CDAP SDK directory, use the Command Line Interface:

    $ cdap cli delete app HelloWorld
    
    > cdap cli delete app HelloWorld