Rebuilding Aggregates Using the REST API

You can use the aggregate-batch endpoint of the AtScale engine REST API to trigger an initial build or rebuild for all aggregates of a published cube. The Design Center provides a sample Bash script that runs curl commands to authenticate with and post a request to the aggregate-batch endpoint for the cube.

One use case for this feature is the need to rebuild aggregates after ETL (Extract, Transform, Load) jobs. Many companies have ETL jobs that they routinely run in Hadoop to cleanse raw data and load that data into Hive. You can use AtScale's REST API to programmatically rebuild aggregates for a cube as a part of your ETL processing.

Before you begin

• The user connecting to the API to trigger the aggregate build must have the "Manage Aggregates" permission. Assign the user the "Manage Aggregates" permission by creating a role with this permission present, or assigning the user an existing role that has this permission enabled.
• The user accessing the Build tab should have the Login, View Aggregates, and Publish Projects permissions. For more information, see Creating and Editing Roles.

Trigger an aggregate build via API

1. At the top of the Design Center, click Projects.
2. In the list of published projects, select the project containing the cube you want to rebuild aggregates for.
3. Select the published version of the cube associated with the project, and select the Build tab. From the drop-down menu, select Triggers. The sample Bash script appears in the API Trigger Command dialog.

Detail	Description
Endpoint	aggregate-batch/org_id/project_id
URL Parameters	org_id: The AtScale organization ID project_id: The AtScale project ID
Query Parameters	cubeID: The AtScale cube ID
HTTP Method	POST
Authentication	Each API call to the AtScale engine must pass in the client-api-key field in the HTTP request header. The client API key value for your installation is shown in the example request.
Return Responses	201 - Created / Success 400 - Bad Request 401 - Unauthorized 404 - Not Found 500 - Server Error 503 - Server Unavailable

Trigger an incremental aggregate build via API

You can also use the API to trigger incremental aggregate builds for specific datasets. This enables you to run ad-hoc incremental builds without having to modify the datasets’ configured grace periods.

Prerequisites

Before performing this procedure, ensure that:

• Grace period overrides are enabled both in the engine and on the cubes you want to run aggregate builds for:

  • Set the aggregate.batch.gracePeriodOverrides.enabled engine setting to true. This setting is disabled by default. You do not need to restart the engine after changing it. For more information, see Settings for Enabling and Configuring Incremental Builds.
  • Set the aggregate.batch.cube.gracePeriodOverrides.enabled setting to true on the cubes you want to run incremental builds on. This setting is disabled by default. For more information, see Aggregate Incremental Builds Settings.

• Incremental builds are enabled on the datasets you want to run aggregate builds on. For more information, see Setting Properties to Allow Incremental Rebuilds of Aggregates.

• You have permission to run incremental builds. For more information, see Performing Full Rebuilds of Incremental Aggregates.

Important

Enabling this functionality can potentially cause your system to become strained by expensive aggregate rebuilds.

Procedure

To run an incremental aggregate build via API:

1. At the top of Design Center, click Projects.
2. In the list of published projects, select the project containing the cube you want to run an incremental aggregate build for.
3. Select the published version of the cube associated with the project, then select the Build tab.
4. In the drop-down menu, select Triggers. The sample Bash script appears in the API Trigger Command dialog.

Detail	Description
Endpoint	aggregate-batch/org_id/project_id
URL Parameters	org_id: The AtScale organization ID project_id: The AtScale project ID
Query Parameters	cubeID: The AtScale cube ID
HTTP Method	POST
Body	grace-period-overrides: A list of the datasets you want to provide grace period overrides for, as well as the numeric override values. You can include as many datasets as you want.
Authentication	Each API call to the AtScale engine must pass in the client-api-key field in the HTTP request header. The client API key value for your installation is shown in the example request.
Return Responses	201 - Created / Success 400 - Bad Request 401 - Unauthorized 404 - Not Found 500 - Server Error 503 - Server Unavailable

What to do next

After an initial build or rebuild is triggered, you can go to the Aggregates > Build History page to view the results. You can also open this page from the menu item View History in the menu for a cube.

This page shows whether builds are queued, running, or successful. It also shows whether builds failed.

If a build is running, you can cancel it, if you need to, by clicking Cancel Build in the entry for the build.

In an entry for a build, you can click the View Instances link to find out information about the instances that were or were not built.

Additional information

How Aggregate Tables Are Populated With Data