Installation on Microsoft Azure HDInsight


This section describes installing CDAP on Microsoft Azure HDInsight using Microsoft's website to:

Information on Microsoft Azure HDInsight is available online.

CDAP 5.1 is compatible with Microsoft Azure HDInsight 3.4, 3.5, and 3.6.

Creating a Cluster with CDAP Installed

  1. Log onto Microsoft Azure and navigate to HDInsight clusters (either through the New menu and under Intelligence + analytics, or directly through the HDInsight clusters icon in the left sidebar):


    Microsoft Azure: Finding HDInsight

  2. Create an HDInsight cluster by clicking the "Create" button. Quick create will create a cluster, but without CDAP installed. We recommend instead using the Custom (size, settings, apps) tab to access additional options and add installing CDAP as part of the cluster creation.


    Microsoft Azure HDInsight: Creating a cluster using Quick create

    Using the Custom (size, settings, apps) tab, enter a Cluster name (such as cdap-cluster). Under Cluster type, select "HBase". (To continue, click Select to close the Cluster configuration window):


    Microsoft Azure HDInsight: Creating a cluster using Custom (size, settings, apps) settings

  3. Enter the required basic settings: Cluster login password, Resource group, and Location:


    Microsoft Azure HDInsight: Configuring basic settings

  4. Under Storage, set a storage account, either an existing account or enter a unique name for a new account.


    Microsoft Azure HDInsight: Setting a storage account

  5. Under Applications, search for CDAP and select the version you'd like to install and accept the legal terms:


    Microsoft Azure HDInsight: Setting CDAP as an application on the cluster

  6. Under Cluster size, at a minimum, a CDAP cluster requires four D3 v2 region nodes. This is enough YARN capacity to run only simple examples. For any serious work, you will need a cluster with additional nodes (Microsoft recommends additional smaller nodes versus fewer larger nodes due to the nature of blob storage):


    Microsoft Azure HDInsight: Setting cluster size

  7. Under Advanced settings, there are currently no settings required:


    Microsoft Azure HDInsight: Advanced settings are currently not required

  8. Click Next to review the summary. Then click Create to start cluster initialization:


    Microsoft Azure HDInsight: Summary and confirming configuration

  9. Once the cluster is ready—it may take a few minutes while the YARN containers come up—its icon will show on the portal dashboard:


    Microsoft Azure HDInsight: Cluster is created and running

  10. Open the created cluster. Click the Applications button to see the installed applications:


    Microsoft Azure HDInsight: The cluster dashboard, Applications button circled

  11. CDAP should show as installed successfully, with a Portal link to access it:


    Microsoft Azure HDInsight: Installed applications, showing CDAP and its Portal link

Install CDAP as an Application on an Existing Cluster

You can add CDAP to an existing cluster. (If you added CDAP in the previous steps for creating a cluster, you can jump to verification.)

  1. Open the created HBase Cluster. Click the Applications button:


    Microsoft Azure HDInsight: The cluster dashboard

  2. In the open window, click + Add to show the Available applications. Choose the CDAP version you'd like to install and accept the legal terms. Once confirmed in the following step, installation of CDAP starts:


    Microsoft Azure HDInsight: Adding CDAP as an application

  3. The initial status starts at Accepted:


    Microsoft Azure HDInsight: CDAP accepted as an application

  4. Once CDAP is installed successfully, its status shows as Installed, with a Portal link to access it:


    Microsoft Azure HDInsight: CDAP installed as an application and its Portal link


  1. Click the Portal link to launch the login window. Login by entering the cluster login name (default admin) and password which are entered when you created the cluster:


    Microsoft Azure HDInsight: Login window

  2. A new window will open with the CDAP "Welcome" page:


    Microsoft Azure HDInsight: CDAP UI "Welcome" page

  3. Navigate to the Administration page from CDAP pulldown menu in the upper-right. (You can also directly enter the URL for the page, https://<cdap-host>:443/cdap/administration, substituting for <cdap-host> the host name of the CDAP server, as shown above in step 1 of Verification):


    Microsoft Azure HDInsight: CDAP Administration page, showing CDAP up and running