Lotus (Lotus v1.0.0)

Copy Markdown View Source

Lotus is a lightweight Elixir library for saving and executing read-only queries against SQL and non-SQL data sources alike.

This module provides the main public API, orchestrating between:

  • Storage: Query persistence and management
  • Runner: statement execution with safety checks
  • Migrations: Database schema management

Configuration

Add to your config:

config :lotus,
  storage_repo: MyApp.Repo,
  default_source: "main",
  data_sources: %{"main" => MyApp.Repo}

Usage

# Create and save a query with variables
{:ok, query} = Lotus.create_query(%{
  name: "Active Users",
  statement: "SELECT * FROM users WHERE active = {{is_active}}",
  variables: [
    %{name: "is_active", type: :text, label: "Is Active", default: "true"}
  ],
  search_path: "reporting, public"
})

# Execute a saved query
{:ok, results} = Lotus.run_query(query)

# Execute a statement directly (read-only)
{:ok, results} = Lotus.run_statement("SELECT * FROM products WHERE price > $1", [100])

Further reading

  • Source adapters guide — how the adapter contract works, building custom SQL dialects or non-Ecto adapters, AI ai_context + trust boundary, security boundaries around variable substitution and visibility.
  • Upgrading to v1.0 — step-by-step migration from v0.x (config renames, DB column rename, middleware/telemetry payload changes, adapter-contract updates).

Summary

Functions

Checks if a query can be run with the provided variables.

Creates a new dashboard.

Creates a new card for a dashboard.

Creates a new filter for a dashboard.

Creates a filter mapping connecting a dashboard filter to a card's query variable.

Creates a new saved query.

Creates a new visualization for a query.

Returns all configured data sources.

Returns the default data source as a {name, module} tuple.

Deletes a dashboard.

Deletes a filter mapping.

Deletes a saved query.

Deletes a visualization (by struct or id).

Describes a specific table, returning its column definitions.

Disables public sharing for a dashboard.

Enables public sharing for a dashboard by generating a unique token.

Gets a single dashboard by ID. Returns nil if not found.

Gets a single dashboard by ID. Raises if not found.

Gets a dashboard by its public sharing token.

Gets a single card by ID. Returns nil if not found.

Gets a single card by ID. Raises if not found.

Gets a single filter by ID. Returns nil if not found.

Gets a single filter by ID. Raises if not found.

Gets a data source by name.

Gets a single query by ID. Returns nil if not found.

Gets a single query by ID. Raises if not found.

Gets statistics for a specific table.

Invalidates all cached discovery entries associated with the given scope.

Lists all filter mappings for a card.

Lists all cards for a dashboard.

Lists all filters for a dashboard.

Lists all dashboards.

Lists dashboards with optional filtering.

Lists the names of all configured data sources.

Lists all saved queries.

Lists all relations (tables with column information) in a data repository.

Lists all schemas in the given repository.

Lists all tables in a data repository.

Lists all visualizations for a query.

Returns the configured Ecto repository where Lotus stores query definitions.

Runs all query cards in a dashboard and returns their results.

Runs a single dashboard card and returns its result.

Run a saved query (by struct or id).

Run an ad-hoc statement (bypassing storage), read-only by default and sandboxed.

Returns whether unique query names are enforced.

Updates an existing dashboard.

Updates an existing query.

Updates an existing visualization.

Validates a visualization config against query results.

Returns the current version of Lotus.

Types

cache_opt()

@type cache_opt() ::
  :bypass
  | :refresh
  | {:ttl_ms, non_neg_integer()}
  | {:profile, atom()}
  | {:tags, [binary()]}

opts()

@type opts() :: [
  timeout: non_neg_integer(),
  statement_timeout_ms: non_neg_integer(),
  read_only: boolean(),
  search_path: binary() | nil,
  repo: binary() | module() | nil,
  vars: map(),
  cache: [cache_opt()] | :bypass | :refresh | nil,
  window: window_opts(),
  filters: [Lotus.Query.Filter.t()],
  sorts: [Lotus.Query.Sort.t()],
  context: term(),
  scope: term()
]

window_count_mode()

@type window_count_mode() :: :none | :exact

window_opts()

@type window_opts() :: [
  limit: pos_integer(),
  offset: non_neg_integer(),
  count: window_count_mode()
]

Functions

can_run?(query, opts \\ [])

@spec can_run?(Lotus.Storage.Query.t(), opts()) :: boolean()

Checks if a query can be run with the provided variables.

Returns true if all required variables have values (either from defaults or supplied vars), false otherwise.

Examples

# Query with all required variables having defaults
Lotus.can_run?(query)
# => true

# Query missing required variables
Lotus.can_run?(query)
# => false

# Query with runtime variable overrides
Lotus.can_run?(query, vars: %{"user_id" => 123})
# => true (if user_id was the missing variable)

child_spec(opts)

create_dashboard(attrs)

Creates a new dashboard.

create_dashboard_card(dashboard_or_id, attrs)

Creates a new card for a dashboard.

create_dashboard_filter(dashboard_or_id, attrs)

Creates a new filter for a dashboard.

create_filter_mapping(card, filter, variable_name, opts \\ [])

Creates a filter mapping connecting a dashboard filter to a card's query variable.

create_query(attrs)

Creates a new saved query.

create_visualization(query_or_id, attrs)

Creates a new visualization for a query.

data_sources()

Returns all configured data sources.

default_data_source()

Returns the default data source as a {name, module} tuple.

  • If there's only one data source configured, returns it
  • If multiple sources are configured and default_source is set, returns that source
  • If multiple sources are configured without default_source, raises an error
  • If no data sources are configured, raises an error

delete_dashboard(dashboard)

Deletes a dashboard.

delete_dashboard_card(card_or_id)

Deletes a card.

delete_dashboard_filter(filter_or_id)

Deletes a filter.

delete_filter_mapping(mapping_or_id)

Deletes a filter mapping.

delete_query(query)

Deletes a saved query.

delete_visualization(viz_or_id)

Deletes a visualization (by struct or id).

describe_table(repo_or_name, table_name, opts \\ [])

Describes a specific table, returning its column definitions.

Options

  • :context — opaque value threaded into the :after_describe_table and :after_discover middleware events.
  • :scope — opaque value passed to the visibility resolver and hashed into the cache key. See Lotus.Visibility.Resolver.

Examples

{:ok, columns} = Lotus.describe_table("primary", "users")
{:ok, columns} = Lotus.describe_table("postgres", "customers", schema: "reporting")
{:ok, columns} = Lotus.describe_table(MyApp.DataRepo, "products", search_path: "analytics, public")

disable_public_sharing(dashboard)

Disables public sharing for a dashboard.

enable_public_sharing(dashboard)

Enables public sharing for a dashboard by generating a unique token.

get_dashboard(id)

Gets a single dashboard by ID. Returns nil if not found.

get_dashboard!(id)

Gets a single dashboard by ID. Raises if not found.

get_dashboard_by_token(token)

Gets a dashboard by its public sharing token.

get_dashboard_card(id, opts \\ [])

Gets a single card by ID. Returns nil if not found.

Options

  • :preload - A list of associations to preload

get_dashboard_card!(id, opts \\ [])

Gets a single card by ID. Raises if not found.

Options

  • :preload - A list of associations to preload

get_dashboard_filter(id)

Gets a single filter by ID. Returns nil if not found.

get_dashboard_filter!(id)

Gets a single filter by ID. Raises if not found.

get_data_source!(name)

Gets a data source by name.

Raises if the source is not configured.

get_query(id)

Gets a single query by ID. Returns nil if not found.

get_query!(id)

Gets a single query by ID. Raises if not found.

get_table_stats(repo_or_name, table_name, opts \\ [])

Gets statistics for a specific table.

Examples

{:ok, stats} = Lotus.get_table_stats("primary", "users")
{:ok, stats} = Lotus.get_table_stats("postgres", "customers", schema: "reporting")
# Returns %{row_count: 1234}

invalidate_scope(scope)

Invalidates all cached discovery entries associated with the given scope.

Uses tag-based invalidation — each scoped cache entry is tagged with a scope digest, so this clears only entries for the specified scope without flushing the entire cache.

Examples

:ok = Lotus.invalidate_scope(%{tenant_id: 42})
:ok = Lotus.invalidate_scope(%{role: :admin})

list_card_filter_mappings(card_or_id)

Lists all filter mappings for a card.

list_dashboard_cards(dashboard_or_id, opts \\ [])

Lists all cards for a dashboard.

Options

  • :preload - A list of associations to preload (e.g., [:query, :filter_mappings])

list_dashboard_filters(dashboard_or_id)

Lists all filters for a dashboard.

list_dashboards(opts \\ [])

Lists all dashboards.

Options

  • :preload - A list of associations to preload (e.g., [:cards])

list_dashboards_by(opts)

Lists dashboards with optional filtering.

Options

  • :search - Search term to match against dashboard names

list_data_source_names()

Lists the names of all configured data sources.

Useful for building UI dropdowns.

list_queries()

Lists all saved queries.

list_relations(repo_or_name, opts \\ [])

Lists all relations (tables with column information) in a data repository.

Options

  • :context — opaque value threaded into the :after_list_relations and :after_discover middleware events.
  • :scope — opaque value passed to the visibility resolver and hashed into the cache key. See Lotus.Visibility.Resolver.

Examples

{:ok, relations} = Lotus.list_relations("postgres", search_path: "reporting, public")
# Returns [{"reporting", "customers"}, {"public", "users"}, ...]

list_schemas(repo_or_name, opts \\ [])

Lists all schemas in the given repository.

Returns a list of schema names. For databases without schemas (like SQLite), returns an empty list.

Options

  • :context — opaque value threaded into the :after_list_schemas and :after_discover middleware events.
  • :scope — opaque value passed to the visibility resolver and hashed into the cache key. See Lotus.Visibility.Resolver.

Examples

{:ok, schemas} = Lotus.list_schemas("postgres")
# Returns ["public", "reporting", ...]

{:ok, schemas} = Lotus.list_schemas("sqlite")
# Returns []

list_tables(repo_or_name, opts \\ [])

Lists all tables in a data repository.

For databases with schemas (PostgreSQL), returns {schema, table} tuples. For databases without schemas (SQLite), returns just table names as strings.

Options

  • :context — opaque value threaded into the :after_list_tables and :after_discover middleware events. See Lotus.Middleware.
  • :scope — opaque value passed to the visibility resolver and hashed into the cache key. Different scopes produce independent cached entries. See Lotus.Visibility.Resolver.

Examples

{:ok, tables} = Lotus.list_tables("postgres")
# Returns [{"public", "users"}, {"public", "posts"}, ...]

{:ok, tables} = Lotus.list_tables("postgres", search_path: "reporting, public")
# Returns [{"reporting", "customers"}, {"public", "users"}, ...]

{:ok, tables} = Lotus.list_tables("sqlite")
# Returns ["products", "orders", "order_items"]

{:ok, tables} = Lotus.list_tables("postgres", context: %{tenant: "acme"})
# Middleware sees `%{tenant: "acme"}` in the payload

{:ok, tables} = Lotus.list_tables("postgres", scope: %{role: :admin})
# Visibility resolver receives scope; result cached separately per scope

list_visualizations(query_or_id)

Lists all visualizations for a query.

Returns visualizations ordered by position, then by id.

reorder_dashboard_cards(dashboard_or_id, card_ids)

Reorders cards in a dashboard.

repo()

Returns the configured Ecto repository where Lotus stores query definitions.

run_dashboard(dashboard_or_id, opts \\ [])

Runs all query cards in a dashboard and returns their results.

Returns a map of card IDs to their results.

Options

  • :filter_values - Map of filter names to their current values
  • :parallel - Whether to run cards in parallel (default: true)
  • :timeout - Timeout per card in milliseconds (default: 30000)

run_dashboard_card(card_or_id, opts \\ [])

Runs a single dashboard card and returns its result.

Options

  • :filter_values - Map of filter names to their current values
  • :timeout - Query timeout in milliseconds

run_query(query_or_id, opts \\ [])

@spec run_query(Lotus.Storage.Query.t() | term(), opts()) ::
  {:ok, Lotus.Result.t()} | {:error, term()}

Run a saved query (by struct or id).

Variables in the query statement (using {{variable_name}} syntax) are substituted with values from the query's default variables and any runtime overrides provided via the vars option.

Variable Resolution

Variables are resolved in this order:

  1. Runtime values from vars option (highest priority)
  2. Default values from the query's variable definitions
  3. If neither exists, raises an error for missing required variable

Examples

# Run query with default variable values
Lotus.run_query(query)

# Override variables at runtime
Lotus.run_query(query, vars: %{"min_age" => 25, "status" => "active"})

# Run with timeout and repo options
Lotus.run_query(query, timeout: 10_000, repo: MyApp.DataRepo)

# Run by query ID
Lotus.run_query(query_id, vars: %{"user_id" => 123})

Variable Types

Variables are automatically cast based on their type definition:

  • :text - Used as-is (strings)
  • :number - Cast from string to integer
  • :date - Cast from ISO8601 string to Date struct

Windowed pagination

Pass window: [limit: pos_integer, offset: non_neg_integer, count: :none | :exact] to return only a page of rows from the original query. When count: :exact, Lotus will also compute SELECT COUNT(*) FROM (original_sql) and include meta.total_count in the result. The num_rows field always reflects the number of rows in the returned page.

run_statement(statement, params \\ [], opts \\ [])

@spec run_statement(
  Lotus.Query.Statement.body(),
  Lotus.Query.Statement.params(),
  opts()
) ::
  {:ok, Lotus.Result.t()} | {:error, term()}

Run an ad-hoc statement (bypassing storage), read-only by default and sandboxed.

The statement is the adapter-native payload: SQL text for Ecto-backed sources, a decoded JSON object or DSL term for others.

Options

  • :read_only — when true (default), blocks write operations (INSERT, UPDATE, DELETE, DDL) at both the application and database level. Set to false to allow write queries.

Examples

# Run against default configured repo
{:ok, result} = Lotus.run_statement("SELECT * FROM users")

# Run against specific repo
{:ok, result} = Lotus.run_statement("SELECT * FROM products", [], repo: MyApp.DataRepo)

# With parameters
{:ok, result} = Lotus.run_statement("SELECT * FROM users WHERE id = $1", [123])

# With search_path for schema resolution
{:ok, result} = Lotus.run_statement("SELECT * FROM users", [], search_path: "reporting, public")

# Allow write queries (development use)
{:ok, result} = Lotus.run_statement(
  "INSERT INTO notes (body) VALUES ($1)",
  ["hello"],
  read_only: false
)

Windowed pagination

Pass window: [limit: pos_integer, offset: non_neg_integer, count: :none | :exact] to page results from the statement. See run_query/2 for details. The cache key automatically incorporates the window so different pages are cached independently.

start_link(opts \\ [])

unique_names?()

Returns whether unique query names are enforced.

update_dashboard(dashboard, attrs)

Updates an existing dashboard.

update_dashboard_card(card, attrs)

Updates a card.

update_dashboard_filter(filter, attrs)

Updates a filter.

update_query(query, attrs)

Updates an existing query.

update_visualization(viz, attrs)

Updates an existing visualization.

validate_visualization_config(config, result)

Validates a visualization config against query results.

Checks that all referenced fields exist in the result columns and that numeric aggregations (sum, avg) are applied only to numeric columns.

version()

Returns the current version of Lotus.