Skip to main content

2.6.0 release notes

This release of AI-Link expands python control of the semantic layer, while improving performance and ease of use for existing functions:

  • Expanded CRUD operation support for datasets and user defined aggregates
  • Enabled a workflow to support OAuth or SAML authentication
  • Consolidated dataset creation functions and TableExists enums into single entities to improve user experience
  • Provided alternate workflow for all utils based functionality. All utils modules/functions are now private

Please refer to our API documentation for the latest syntax to use with AI-Link. See below for updates associated with this release.

User Flow Improvements​

  • The PandasTableExistsActionType and PysparkTableExistsActionType enums have been consolidated into TableExistsAction, which may impact existing code
  • The add_query_dataset and update_query_dataset functions were removed in place of using the more general add_dataset and update_dataset
  • The update_secondary_attribute function was removed in place of the more general update_categorical_feature function
  • The generate_db_query function was removed in place of data_model.get_database_query

New Python Helper Functions for Programmatic Interaction​

  • CRUD operation support: added additional functions create and update various objects in the semantic layer
  • Added function parameters: additional parameter were added to dataset related functions to allow better configuration

Non-Functional Updates​

  • Performance: Data fetch speed for Databricks submit_query function was improved
  • Bug fixes: bug fixes addressing roleplaying and features with different key/value columns when joining objects to the semantic layer

Changelog for Syntax Updates​

data_model.py::DataModel​

NEW FUNCTIONS:

  • create_user_defined_aggregate

    • Creates a user defined aggregate containing the given features.

  • update_categorical_feature

    • Updates the metadata for a categorical feature.

  • update_dataset

    • Updates the settings for a dataset.

  • create_dataset

    • creates a dataset for the given table or query

  • validate_mdx

    • validates is an MDX expression is valid for the current data model

UPDATED FUNCTIONS:

  • create_dataset

    • Added optional parameter dimension_only

      • marks the dataset to only be used for dimensions so it will not be placed on the main canvas

    • Added optional parameter incremental_indicator

      • the column to use as an indicator for incremental aggregate updates

    • Added optional parameter grace_period

      • the grace period for incremental aggregate updates

    • Added optional parameter safe_to_join_to_incremental

      • whether the dataset is static and safe to join to datasets being incrementally updated

  • get_dataset

    • Added new metadata fields to the response: create_hinted_aggregate, safe_to_join_to_incremental, incremental_indicator, grace_period

  • update_dataset

    • Added optional parameter incremental_indicator

      • the column to use as an indicator for incremental aggregate updates

    • Added optional parameter grace_period

      • the grace period for incremental aggregate updates

    • Added optional parameter safe_to_join_to_incremental

      • whether the dataset is static and safe to join to datasets being incrementally updated

    • Added optional parameter create_fact_from_dimension

      • updates a dimension only dataset to allow it to be used as a fact dataset and adds it to the main canvas

  • get_features

    • Fixed issue with atscale_type not correctly displaying aggregation method for measures

  • get_database_query

    • Improved performance in retrieving SQL from the engine

  • writeback_spark

    • Fixed datatype check.

  • writeback_spark_jdbc

    • Fixed datatype check.

  • dataset_exists

    • Added optional parameter include_unused
      • allows users to see datasets available in the project but not used by the current data model

REMOVED FUNCTIONS:

  • add_query_dataset

    • Deprecated in place of create_dataset

  • update_secondary_attribute

    • Replaced by update_categorical_feature which can update both levels and secondary attributes.

  • update_query_dataset

    • Replaced by update_dataset which can update both query and table based datasets.

enums.py​

NEW FUNCTIONS:

  • TableExistsAction
    • A generalized class to take the place of the similar pandas and pyspark action type classes

REMOVED FUNCTIONS:

  • PandasTableExistsActionType and PysparkTableExistsActionType
    • Replaced with a more general class TableExistsAction

sql_connection.py::SQLConnection​

NEW FUNCTIONS:

  • execute_statement
    • Executes a single SQL statement using the database connection.

databricks.py::Databricks​

UPDATED FUNCTIONS:

  • submit_query
    • Improved fetch performance

feature_engineering.py​

UPDATED FUNCTIONS:

  • generate_time_series_features
    • Fixed interaction between groupbys and shifts when doing rolling metrics.

connection.py::Connection​

UPDATED FUNCTIONS:

  • _auth
    • Updated authentication if any OAuth or SAML is set up for the organization so the user will be prompted to retrieve a token from the browser.

query_utils.py​

REMOVED FUNCTIONS:

  • generate_db_query
    • Deprecated in favor of data_model.get_database_query