bi_etl.database.database_metadata module

Created on Dec 23, 2015

@author: Derek Wood

class bi_etl.database.database_metadata.DatabaseMetadata(bind=None, reflect=False, schema=None, quote_schema=None, naming_convention={'ix': 'ix_%(column_0_label)s'}, info=None, database_name=None, uses_bytes_length_limits=None)

Bases: MetaData

A light wrapper over sqlalchemy.schema.MetaData

__init__(bind=None, reflect=False, schema=None, quote_schema=None, naming_convention={'ix': 'ix_%(column_0_label)s'}, info=None, database_name=None, uses_bytes_length_limits=None)

Create a new MetaData object.

Parameters:

• schema –

  The default schema to use for the _schema.Table, Sequence, and potentially other objects associated with this _schema.MetaData. Defaults to None.

  See also

  Specifying a Default Schema Name with MetaData - details on how the _schema.MetaData.schema parameter is used.

  _schema.Table.schema

  Sequence.schema

• quote_schema – Sets the quote_schema flag for those _schema.Table, Sequence, and other objects which make usage of the local schema name.

• info – Optional data dictionary which will be populated into the SchemaItem.info attribute of this object.

• naming_convention –

  a dictionary referring to values which will establish default naming conventions for Constraint and Index objects, for those objects which are not given a name explicitly.

  The keys of this dictionary may be:

  • a constraint or Index class, e.g. the UniqueConstraint, _schema.ForeignKeyConstraint class, the Index class

  • a string mnemonic for one of the known constraint classes; "fk", "pk", "ix", "ck", "uq" for foreign key, primary key, index, check, and unique constraint, respectively.

  • the string name of a user-defined “token” that can be used to define new naming tokens.

  The values associated with each “constraint class” or “constraint mnemonic” key are string naming templates, such as "uq_%(table_name)s_%(column_0_name)s", which describe how the name should be composed. The values associated with user-defined “token” keys should be callables of the form fn(constraint, table), which accepts the constraint/index object and _schema.Table as arguments, returning a string result.

  The built-in names are as follows, some of which may only be available for certain types of constraint:

  • %(table_name)s - the name of the _schema.Table object associated with the constraint.

  • %(referred_table_name)s - the name of the _schema.Table object associated with the referencing target of a _schema.ForeignKeyConstraint.

  • %(column_0_name)s - the name of the _schema.Column at index position “0” within the constraint.

  • %(column_0N_name)s - the name of all _schema.Column objects in order within the constraint, joined without a separator.

  • %(column_0_N_name)s - the name of all _schema.Column objects in order within the constraint, joined with an underscore as a separator.

  • %(column_0_label)s, %(column_0N_label)s, %(column_0_N_label)s - the label of either the zeroth _schema.Column or all Columns, separated with or without an underscore

  • %(column_0_key)s, %(column_0N_key)s, %(column_0_N_key)s - the key of either the zeroth _schema.Column or all Columns, separated with or without an underscore

  • %(referred_column_0_name)s, %(referred_column_0N_name)s %(referred_column_0_N_name)s, %(referred_column_0_key)s, %(referred_column_0N_key)s, … column tokens which render the names/keys/labels of columns that are referenced by a _schema.ForeignKeyConstraint.

  • %(constraint_name)s - a special key that refers to the existing name given to the constraint. When this key is present, the Constraint object’s existing name will be replaced with one that is composed from template string that uses this token. When this token is present, it is required that the Constraint is given an explicit name ahead of time.

  • user-defined: any additional token may be implemented by passing it along with a fn(constraint, table) callable to the naming_convention dictionary.

  Added in version 1.3.0: - added new %(column_0N_name)s, %(column_0_N_name)s, and related tokens that produce concatenations of names, keys, or labels for all columns referred to by a given constraint.

  See also

  Configuring Constraint Naming Conventions - for detailed usage examples.

begin(connection_name: str = None) → Transaction

clear() → None

Clear all Table objects from this MetaData.

close_connection(connection_name: str = None)

close_connections(exceptions: set = None)

commit(connection_name: str = None)

Commit based on a connection name rather than via a ‘sqlalchemy.engine.base.Transaction’ object (which you could call .commit() on

Parameters:

connection_name

connect(connection_name: str = None) → Connection

connection(connection_name: str = None, open_if_not_exist: bool = True, open_if_closed: bool = True) → Connection

create_all(bind: _CreateDropBind, tables: _typing_Sequence[Table] | None = None, checkfirst: bool = True) → None

Create all tables stored in this metadata.

Conditional by default, will not attempt to recreate tables already present in the target database.

Parameters:

• bind – A Connection or Engine used to access the database.

• tables – Optional list of Table objects, which is a subset of the total tables in the MetaData (others are ignored).

• checkfirst – Defaults to True, don’t issue CREATEs for tables already present in the target database.

create_drop_stringify_dialect = 'default'

property dialect

property dialect_name

dispatch = <sqlalchemy.event.base.DDLEventsDispatch object>

dispose()

This method leaves the possibility of checked-out connections remaining open, as it only affects connections that are idle in the pool.

drop_all(bind: _CreateDropBind, tables: _typing_Sequence[Table] | None = None, checkfirst: bool = True) → None

Drop all tables stored in this metadata.

Conditional by default, will not attempt to drop tables not present in the target database.

Parameters:

• bind – A Connection or Engine used to access the database.

• tables – Optional list of Table objects, which is a subset of the total tables in the MetaData (others are ignored).

• checkfirst – Defaults to True, only issue DROPs for tables confirmed to be present in the target database.

drop_table_if_exists(table_name, schema=None, connection_name: str = None, transaction: bool = False, auto_close: bool = False)

execute(sql, *list_params, transaction: bool = True, auto_close: bool = True, connection_name: str = None, **params)

execute_direct(sql, return_results=False)

execute_procedure(procedure_name, *args, return_results=False, dpapi_connection=None)

Execute a stored procedure

Parameters:

• procedure_name (str) – The procedure to run.

• args – The arguments to pass

• return_results – Needs to be a keyword param. Should we try and get result rows from the procedure.

• dpapi_connection – A raw dpapi connection to use. Optional.

Raises:

• sqlalchemy.exc.DBAPIError: – API error

• sqlalchemy.exc.DatabaseError: – Proxy for database error

has_active_transaction(connection_name: str = None) → bool

info

Info dictionary associated with the object, allowing user-defined data to be associated with this SchemaItem.

The dictionary is automatically generated when first accessed. It can also be specified in the constructor of some objects, such as _schema.Table and _schema.Column.

is_connected(connection_name: str = None) → bool

static qualified_name(schema, table)

reflect(bind: Engine | Connection, schema: str | None = None, views: bool = False, only: _typing_Sequence[str] | Callable[[str, MetaData], bool] | None = None, extend_existing: bool = False, autoload_replace: bool = True, resolve_fks: bool = True, **dialect_kwargs: Any) → None

Load all available table definitions from the database.

Automatically creates Table entries in this MetaData for any table available in the database but not yet present in the MetaData. May be called multiple times to pick up tables recently added to the database, however no special action is taken if a table in this MetaData no longer exists in the database.

Parameters:

• bind – A Connection or Engine used to access the database.

• schema – Optional, query and reflect tables from an alternate schema. If None, the schema associated with this _schema.MetaData is used, if any.

• views – If True, also reflect views (materialized and plain).

• only –

  Optional. Load only a sub-set of available named tables. May be specified as a sequence of names or a callable.

  If a sequence of names is provided, only those tables will be reflected. An error is raised if a table is requested but not available. Named tables already present in this MetaData are ignored.

  If a callable is provided, it will be used as a boolean predicate to filter the list of potential table names. The callable is called with a table name and this MetaData instance as positional arguments and should return a true value for any table to reflect.

• extend_existing – Passed along to each _schema.Table as _schema.Table.extend_existing.

• autoload_replace – Passed along to each _schema.Table as _schema.Table.autoload_replace.

• resolve_fks –

  if True, reflect _schema.Table objects linked to _schema.ForeignKey objects located in each _schema.Table. For _schema.MetaData.reflect(), this has the effect of reflecting related tables that might otherwise not be in the list of tables being reflected, for example if the referenced table is in a different schema or is omitted via the MetaData.reflect.only parameter. When False, _schema.ForeignKey objects are not followed to the _schema.Table in which they link, however if the related table is also part of the list of tables that would be reflected in any case, the _schema.ForeignKey object will still resolve to its related _schema.Table after the _schema.MetaData.reflect() operation is complete. Defaults to True.

  Added in version 1.3.0.

  See also

  _schema.Table.resolve_fks

• **dialect_kwargs – Additional keyword arguments not mentioned above are dialect specific, and passed in the form <dialectname>_<argname>. See the documentation regarding an individual dialect at Dialects for detail on documented arguments.

See also

Reflecting Database Objects

_events.DDLEvents.column_reflect() - Event used to customize the reflected columns. Usually used to generalize the types using _types.TypeEngine.as_generic()

Reflecting with Database-Agnostic Types - describes how to reflect tables using general types.

remove(table: Table) → None

Remove the given Table object from this MetaData.

rename_table(schema, table_name, new_table_name)

resolve_connection_name(connection_name: str = None) → str

rollback(connection_name: str = None)

schema

session()

property sorted_tables: List[Table]

Returns a list of _schema.Table objects sorted in order of foreign key dependency.

The sorting will place _schema.Table objects that have dependencies first, before the dependencies themselves, representing the order in which they can be created. To get the order in which the tables would be dropped, use the reversed() Python built-in.

Warning

The MetaData.sorted_tables attribute cannot by itself accommodate automatic resolution of dependency cycles between tables, which are usually caused by mutually dependent foreign key constraints. When these cycles are detected, the foreign keys of these tables are omitted from consideration in the sort. A warning is emitted when this condition occurs, which will be an exception raise in a future release. Tables which are not part of the cycle will still be returned in dependency order.

To resolve these cycles, the _schema.ForeignKeyConstraint.use_alter parameter may be applied to those constraints which create a cycle. Alternatively, the _schema.sort_tables_and_constraints() function will automatically return foreign key constraints in a separate collection when cycles are detected so that they may be applied to a schema separately.

Changed in version 1.3.17: - a warning is emitted when MetaData.sorted_tables cannot perform a proper sort due to cyclical dependencies. This will be an exception in a future release. Additionally, the sort will continue to return other tables not involved in the cycle in dependency order which was not the case previously.

See also

_schema.sort_tables()

_schema.sort_tables_and_constraints()

_schema.MetaData.tables

_reflection.Inspector.get_table_names()

_reflection.Inspector.get_sorted_table_and_fkc_names()

table_inventory(schema=None, force_reload=False)

tables

A dictionary of _schema.Table objects keyed to their name or “table key”.

The exact key is that determined by the _schema.Table.key attribute; for a table with no _schema.Table.schema attribute, this is the same as _schema.Table.name. For a table with a schema, it is typically of the form schemaname.tablename.

See also

_schema.MetaData.sorted_tables

transaction(connection_name: str = None) → Transaction

property uses_bytes_length_limits