Skip to main content

databricks

class atscale.db.connections.databricks.Databricks​

The child class of SQLConnection whose implementation is meant to handle interactions with Databricks.

clear_auth​

Clears any authentication information, like password or token from the connection.

execute_statements​

Executes a list of SQL statements. Does not return any results but may trigger an exception.

  • Parameters: statements_list (list) – a list of SQL statements to execute.

platform_type_str : str = 'databricks'​

The string representation of the platform type, matches with atscale

submit_queries​

Submits a list of queries, collecting the results in a list of dictionaries.

  • Parameters: query_list (list) – a list of queries to submit.
  • Returns: A list of pandas DataFrames containing the results of the queries.
  • Return type: List(DataFrame)

write_df_to_db​

Writes the provided pandas DataFrame into the provided table name. Can pass in if_exists to indicate the intended behavior if : the provided table name is already taken.

  • Parameters:
    • table_name (str) – What table to write the dataframe into
    • dataframe (DataFrame) – The pandas DataFrame to write into the table
    • dtypes (Dict , optional) – the datatypes of the passed dataframe. Keys should match the column names. Defaults to None and type will be text.
    • if_exists (enums.TableExistsAction , optional) – The intended behavior in case of table name collisions. Defaults to enums.TableExistsAction.ERROR.
    • chunksize (int , optional) – the chunksize for the write operation.