4588d989198739fb560ac060a79310e822bf1410718eb8c832406c9211916369
Browser storage getItem callBrowser storage setItem call
SHA-2564588d989198739fb560ac060a79310e822bf1410718eb8c832406c9211916369
MaleculeH₂(FU)
Evidence
1<!DOCTYPE html><html lang="en" data-astro-cid-mkjmkm32> <head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><script>
2 (function () {
3 try {
4 // Cross-site theme handoff: the Documentation dropdown's
5 // cross-product links carry ?theme=light|dark so the choice
6 // follows the user across origins. If present, use it, persist
12 if (q === "dark" || q === "light") {
13 dark = q === "dark";
14 try { localStorage.setItem("docs-theme", q); } catch (e) {}
15 url.searchParams.delete("theme");
16 try {
17 history.replaceState(null, "", url.pathname + url.search + url.hash);
18 } catch (e) {}
19 } else {
20 var stored = localStorage.getItem("docs-theme");
21 dark = stored
22 ? stored === "dark"
31 }
32 })();
33 </script><title>Architecture | Dolt Docs</title><meta property="og:title" content="Architecture | Dolt Documentation"><meta property="og:type" content="website"><meta property="og:site_name" content="Dolt Documentation"><meta property="og:url" content="https://dolthub.com/docs/architecture/architecture/"><meta property="og:image" content="https://dolthub.com/docs/og-image.png"><meta name="twitter:card" content="summary_large_image"><meta name="twitter:title" content="Architecture | Dolt Documentation"><meta name="twitter:image" content="https://dolthub.com/docs/og-image.png"><link rel="icon" type="image/png" href="/docs/favicon.png"><link rel="icon" type="image/x-icon" href="/docs/favicon.ico"><link rel="alternate" type="text/markdown" href="/docs/architecture/architecture.md"><link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Source+Sans+Pro:ital,wght@0,400;0,600;0,700;1,400;1,600;1,700&family=Space+Mono:ital,wght@0,400;0,700;1,400;1,700&display=swap"><link href="/docs/pagefind/pagefin
34<p>Dolt is the world’s first version controlled SQL database.</p>
35<p>How would you build a version controlled SQL database? We will start with some of the high level requirements, explain some possible architecture options, and then explain the actual building blocks used to build Dolt.</p>
36<p><img src="/docs/_astro/architecture-icon.c9si9NM7_Z170scn.webp" alt loading="lazy" decoding="async" width="256" height="256"></p>
37<h2 id="requirements">Requirements<a class="anchor-link" aria-label="Link to heading" href="#requirements">#</a></h2>
38<p>Dolt started with the idea that data was too hard to share. “Share” is a loaded word here. Share in this context means distribute, collaborate on, change, buy, and sell.</p>
39<p>Most data was and still is shared as comma separated value (CSV) files. CSVs are easy to build and consume but hard to change. If you need to modify data distributed as a CSV, it becomes a pain to merge those changes back into the “main” copy. You can do it with a tool like Git but the rows and columns must be sorted in the same way as you received them or you will get nonsensical diffs and merge conflicts.</p>
40<p>We thought if we added diff, branch, and merge to data in some way, data would be easier to share. The standard for diff, branch, and merge is Git. We wanted to mimic the Git model of version control for data because almost every developer is at least passingly familiar with Git.</p>
41<p>The more complicated the data, the harder sharing becomes. CSVs map to individual tables. Databases have multiple tables, schema, and constraints to ensure quality of complicated data. Sharing CSVs sacrifices all these tools. You have a collection of files and it is up to you to interpret them. As we got deeper into the data sharing problem, we realized modern databases worked well managing complicated data. We decided the form of data people should share was databases, not tables. Dolt needed to be a full fledged, modern SQL relational database with schema, constraints, and powerful query capabilities in addition to Git-style version control capabilities.</p>
61<p>In order to satisfy the version control requirements of storage compression and fast diff, we knew we would need our own storage engine. There was no way to adapt standard B-tree based database storage engines to the version control task.</p>
62<p>Could Git be adapted to be a storage engine for a database? <a href="https://www.dolthub.com/blog/2021-11-26-so-you-want-git-database/#using-git-as-a-database">Other people have tried</a>. Git storage is not built for fast seek. You really need something close to a B-tree with special version control properties for your storage engine.</p>
63<p>To achieve the SQL database requirements, we looked at the most popular open source options: MySQL and Postgres. Both <a href="https://dev.mysql.com/doc/refman/8.0/en/storage-engines.html">MySQL</a> and <a href="https://www.dolthub.com/blog/2022-01-26-creating-a-postgres-foreign-data-wrapper/">Postgres</a> have pluggable storage engines. But how would we access the version control properties of the storage engine? We needed more control over the supported SQL dialect than MySQL or Postgres exposed.</p>
64<h2 id="oh-god-you-built-a-sql-database-from-scratch">Oh God. You built a SQL database from scratch?!?<a class="anchor-link" aria-label="Link to heading" href="#oh-god-you-built-a-sql-database-from-scratch">#</a></h2>
65<p>Not really. We leveraged three existing open source packages:</p>