d8a.tech (pronounced "data tech") is an open source, warehouse-native web analytics pipeline. It receives the GA4 and Matomo tracking protocols, or its own web tracker, builds sessions on the backend, and writes clean, flat tables to BigQuery, ClickHouse or CSV/Parquet files. Run it in your own infrastructure, air-gapped if you need to, or use d8a.tech Cloud.
flowchart LR
A["GA4 tags (gtag, GTM)<br/>Matomo tracker<br/>d8a web tracker"] --> B["d8a<br/>sessions, attribution,<br/>filters"]
B --> C["BigQuery<br/>ClickHouse<br/>CSV / Parquet on S3, MinIO, GCS or disk"]
C --> D["Your BI, SQL<br/>and AI tools"]
- Keep your tracking. d8a speaks the GA4 and Matomo tracking protocols. Duplicate your GA4 requests to d8a and keep GA4 running, redirect them to d8a instead of Google, or switch to the d8a web tracker, a drop-in replacement for
gtag.js. - Sessions built on the backend. Hits are grouped into sessions server-side, with a configurable inactivity timeout (30 minutes by default) and split rules for campaign changes, user ID changes, maximum duration and maximum events. Each session, with all its events, is written to the warehouse the moment it closes.
- A flat, documented schema. Events and sessions land as flat columns, ready for reporting, with no
UNNEST. Custom parameters can be mapped to their own columns. Every column carries its own documentation, and the schema reference is generated from it. - Raw signals included. Click IDs (
gclid,gbraid,wbraid,dclid,srsltid,fbclid,msclkid) get dedicated columns, and the full landing URL keeps any other click ID queryable. IP addresses can be stored in full or masked at one of four levels. - Attribution and filtering. Source and medium are derived from click IDs, referrers (search, social, email, video and AI assistants) and UTM parameters. Expression-based traffic filters exclude internal traffic or bots, with a test mode.
- Your storage. Write directly to Google BigQuery or ClickHouse, or to CSV or Parquet files on S3, MinIO, GCS or a local filesystem.
- Deploy where your policy requires. Run it on-premises or in your own cloud. The
--airgappedflag disables built-in outbound integrations. - Open source. MIT-licensed, transparent and auditable. Fork it, contribute, or self-host it.
Sessions are built on the backend. Once a visit has been inactive for the session timeout (30 minutes by default, configurable with sessions.timeout), the session closes and all of its events, plus the finished session columns, are written to your warehouse at once. Every row is final when it arrives.
d8a runs in production for healthcare and government organizations. For example, GovAnalytics.nl, a whitelabel analytics platform for Dutch government organizations, is built on d8a.
Typical reasons teams choose it:
- Healthcare. Keep visitor data inside infrastructure you control. d8a.tech signs BAAs.
- Public sector. Self-hosted or air-gapped deployments, with code your auditors can read.
- Data teams and ecommerce. Sessionized, flat tables in your own warehouse, instead of rebuilding sessions from GA4's nested BigQuery export.
Receivers scale out; session processing runs on a single worker that scales vertically. The production operations guide documents roughly 7,000 requests per second (about 7 billion events a month) on a machine with 16 cores and 30 GB of RAM, and about 100 million events a month on 2 CPUs and 4 GB of RAM.
This repository is the full collection and processing pipeline. It ships without a reporting UI: bring your own BI tool, or start from the prebuilt Data Studio dashboard.
d8a.tech Cloud runs the same pipeline in the EU. The Cloud Dashboard plan adds a built-in analytics dashboard with goals, funnels and custom reports, plus Ask AI. The Cloud Warehouse plan writes to your own BigQuery or ClickHouse and comes with the prebuilt Data Studio dashboard. See pricing for the free tiers, the support package and Enterprise options.
The built-in dashboard and Ask AI are part of the d8a.tech Cloud Dashboard plan. They are not included in this repository.
|
|
|
| Dashboard (Cloud only) | Dashboard, light mode (Cloud only) | Ask AI (Cloud only) |
These reports are built on d8a data in Data Studio, using the prebuilt Data Studio dashboard.
|
|
|
| Overview | Location | Ecommerce |
Start for free at app.d8a.tech.
See the Getting started guide and the production operations guide.
- Run it.
go run main.go server --config config.dev.yaml- Send a request and wait about 10 seconds for the session to close (the dev config uses a short session timeout).
curl "http://localhost:8080/g/collect?v=2&tid=14&dl=https%3A%2F%2Ffoo.bar&en=page_view&cid=ag9" -X POST- Check the console output to verify events are being processed.
go test ./...- Getting started
- Database schema and columns
- Traffic attribution
- Technical deep dive
- Glossary of analytics terms
This work is developed with AI assistance, by professional software engineers with more than 10 years of experience in data/analytics space. Every line of code is manually reviewed before it's merged. Vibe-coded PRs made without understanding of the system will be declined.
We welcome contributions from the community. Bug fixes and targeted improvements are always appreciated. For larger refactors or architectural changes, we ask that you discuss your ideas with us on Discord or GitHub Issues first. While modern AI tools make it easy to generate extensive code changes, we need to carefully balance the needs of our community and maintain the project's direction.
You may notice some abstractions in the codebase that seem unnecessary. These exist to support our cloud service, which uses alternative implementations in certain areas.
Please ensure contributions are thoughtful and align with our project standards. PRs that appear to be purely AI-generated without human review and consideration may not be accepted. We value quality over quantity and appreciate the time you take to understand our codebase before contributing.
- Discord
- Website
- Talk to us about deployments, support or partnerships
MIT. See LICENSE for details.