Repository navigation
3 - Avoid recreating snapshot_by_name in evaluator #6019
Description
File: sqlmesh/core/snapshot/evaluator.py
Function: promote , _promote_snapshot
Lines: 267 - 330 , 1253 - 1285
Issue with _promote_snapshot
In _promote_snapshot we are passed the snapshots, which is keyed by snapshot_id. The function needs to key by snapshot_name though, so we keep recreating:
snapshot_by_name = {s.name: s for s in (snapshots or {}).values()}
render_kwargs["snapshots"] = snapshot_by_name
We should just create this once in promote before we pass it in.
# Fetch the view data objects for the promoted snapshots to get them cached
self._get_virtual_data_objects(target_snapshots, environment_naming_info)
snapshot_by_name = {s.name: s for s in (snapshots or {}).values()} <------NEW
deployability_index = deployability_index or DeployabilityIndex.all_deployable()
with self.concurrent_context():
concurrent_apply_to_snapshots(
target_snapshots,
lambda s: self._promote_snapshot(
s,
start=start,
end=end,
execution_time=execution_time,
snapshots=snapshots,
snapshots_by_name=snapshots_by_name, <-------- NEW
table_mapping=table_mapping,
environment_naming_info=environment_naming_info,
deployability_index=deployability_index, # type: ignore
on_complete=on_complete,
),
self.ddl_concurrent_tasks,
)
Roughly. I would also evaluate if we can remove the snapshots input to self._promote_snapshot. Maybe snapshots_by_name fulfills all the functions downstream.
Activity