A new approach to the data analyst profession: end-to-end ownership of analytical data flows

1. Introduction

Data quality, processing speed, and timely delivery are increasingly important to management decisions and the development of analytical systems. Data supports reporting, forecasting, and machine learning. As the number of sources, models, and consumers grows, organizations face greater demands not only on their infrastructure but also on how data specialists work together.

In a conventional enterprise data warehouse team, an analyst investigates source systems, defines transformation rules, prepares SQL mappings, and writes specifications. A data engineer then implements the ingestion and processing logic. This division enables specialization, but it may also require repeated transfers of context, additional coordination, and leave accountability for the finished analytical flow unclear. This problem provides the starting point for this study.

The evolution of data warehouse, data lake, and data lakehouse architectures [1, 2], together with transformation tools such as dbt [3, 4], invites a reassessment of these boundaries. Storage architecture, however, does not dictate an organizational model. Both ETL and ELT can operate in different system architectures, and adopting dbt does not in itself establish end-to-end data ownership.

The research question is: under what conditions might assigning end-to-end responsibility for an analytical data flow to an analytics engineer improve the speed of change, metric consistency, and data quality management compared with a functionally divided workflow? The objective is to propose a responsibility model and criteria for testing it, without assuming that either approach is universally superior.

2. Theoretical background

Machado and Russa [4] describe analytics engineering as an evolution of business intelligence and data modeling practices supported by modern engineering tools. Their book covers the analytics engineer role (Chapter 1), analytical modeling (Chapter 2), SQL (Chapter 3), testing and documentation in dbt (Chapter 4), incremental models and macros (Chapter 5), and an end-to-end analytics example (Chapter 6). These capabilities support reusable business logic and automated validation, but they do not prescribe a single organizational structure for every company.

In this study, an analytical data flow is a connected set of sources, transformations, models, checks, and downstream consumers serving a particular business domain. End-to-end ownership does not require one person to perform every technical task. The flow owner coordinates changes and is accountable for the fitness of the analytical output, while specialized teams retain responsibility for infrastructure.

3. Materials and methods

This is a conceptual and analytical study. It draws on the author's initial problem statement, the technical monograph [4], sources [1‑3], and previously collected public company cases [5‑9]. The method compares responsibilities across the analytical data lifecycle and develops testable hypotheses. The case metrics reflect accompanying technology changes and are therefore not interpreted as causal evidence of end-to-end ownership.

4. Results: proposed model

5. Public case studies and limits of interpretation

Public case studies report lower costs and faster processing following the adoption of modern analytics stacks. For example, vendor case studies for Enpal and Symend report reductions of approximately 70% in selected infrastructure costs or compute consumption, while the Siemens case reports a reduction in one data-loading process from six hours to 25 minutes [5‑7]. These figures come from promotional case studies rather than controlled research. They reflect broader technology changes and do not establish the effect of assigning organizational responsibility.

6. Quantitative analysis of published results

The quantitative comparison includes public cases reporting a before-and-after value for the same metric. Percentage change is calculated as (after − before) / before × 100%. Availability is also expressed as a difference in percentage points. These are descriptive case observations, not findings from a controlled experiment.

Siemens: 360 to 25 minutes (−93.1%); Symend: 720 to 120 minutes (−83.3%). The cases measure different processes, so absolute processing times should not be compared directly between companies. Sources: [7], [6].

Availability increased from 80% to 95% (+15 percentage points; an 18.75% relative increase). This measures availability, not the accuracy of values shown in the reports. Source: [9].

Index: 100 to 30 (−70%). Absolute costs were not disclosed; the chart does not represent dollar savings. Source: [6].

Analysis: Siemens associates the shorter runtime with parallel execution of independent models and incremental processing. Symend reports moving away from full refreshes toward incremental models. At Code42, improved availability accompanied a transition from dbt Core to dbt Cloud, with CI/CD, logging, and alerts. The observed improvements are consistent with engineering practices such as model reuse, testing, and controlled releases, but they do not isolate the causal effect of end-to-end flow ownership.

A direct test of the author's hypothesis requires a pilot using comparable data-mart development tasks before and after changing the responsibility model. Measures should include lead time from approved specification to release, the number of handoffs between roles, time spent fixing defects, and the share of work requiring rework. Without such evidence, a chart comparing "traditional teams" with "analytics engineers" would be misleading.

7. Testable hypotheses and future evaluation

H1: For tasks of comparable complexity and teams of similar composition, end-to-end flow ownership reduces handoffs between specialists.

H2: Shared models and automated checks reduce conflicting KPI definitions.

H3: Integrating development and maintenance of the analytical layer reduces recovery time following source-system changes.

H4: The size of the effect depends on infrastructure complexity, staff capabilities, and data governance maturity.

Evaluation should measure median lead time from request to model release, handoffs per task, incidents per 100 changes, detection and recovery time, processing costs, and the proportion of models with tests and designated owners. A before-and-after comparison must account for concurrent platform migration, changes in workload, and team composition. Until such a study is conducted, the proposed benefits remain hypotheses rather than demonstrated outcomes.

8. Conclusion

This paper proposes the analytical data flow, rather than an individual ETL/ELT stage, as a possible unit of accountability. End-to-end ownership may reduce fragmentation of business context and make changes easier to manage, while preserving the need for data engineering specialization. The technologies discussed in the literature make this arrangement feasible. Public cases demonstrate the potential benefits of modernizing analytics workflows, but do not isolate the contribution of the organizational model. The next step is to test the hypotheses empirically using comparable teams or projects.

9. Declarations

9.1. Conflict of interest

The author declares no conflict of interest.

9.2. Funding

This research received no external funding.

9.3. Data availability

No new data were created in this study. The values in Section 6 are taken from the public case pages [6], [7], and [9].

9.4. Peer review

The manuscript underwent Double-Anonymous Peer Review by at least two independent subject-matter reviewers in accordance with the current editorial policy of IAE Review. The author is not affiliated with the journal's editorial office or publisher; the final publication decision was made by the editorial office.