on GitHub

Tutorial: your first page

By the end of this page you'll know the whole framework. That's not a big claim: there are only four building blocks, and everything else you'll ever make here is a bigger version of one of them. We'll learn all four by building one small thing end to end: a tip predictor that fits a real regression to a table of restaurant checks and redraws every time you drag a slider. You already have everything you need: nothing to install, no database to stand up, no data to download.

Here's the only structural idea, and then we start. A page is two files that share a folder:

  • data.py: Python that prepares tables
  • page.mdx: Markdown that puts them on screen

They're connected by name, and nothing else. When you write data="db.tips", db is the database you create in data.py (db = Database()) and tips is a table you defined on it. No import, no API endpoint, no fetch: just a name that matches.

Each step adds one idea and they stack. The first three stand alone; the fourth ties them together into something live:

StepYou'll writeWhat it gives you
1 · data@db.data()a table on the page
2 · view@tips.view()reshape it with SQL, in the browser
3 · derive@tips.derive()reshape it with pandas, in Python
4 · rpc@db.rpc()recompute live from a slider

Open tutorial/data.py beside this page and read the two files as pairs: section 1 here next to section 1 there. Let's get a table on the screen.

1. Getting data on the page

Everything starts with a table. To make one, write a normal Python function that returns a DataFrame and mark it with @db.data(). The function's name becomes the table's name. This one is called tips:

@db.data()
def tips():
    # one row per party: the bill, the tip, the day, the party size
    return sns.load_dataset("tips")

That's the whole connection. Now switch to the other half: this file, page.mdx. Here you don't write Python; you write component tags, and each one reads a table by name. You say what to measure, never how to count it. Here is the actual markup you'd type into the page:

<Columns cols={3}>
  <Metric label="Parties" data="db.tips" value="count(*)" />
  <Metric
    label="Avg bill"
    data="db.tips"
    value="avg(total_bill)"
    valueFormat="$,.2f"
  />
  <Metric
    label="Avg tip"
    data="db.tips"
    value="avg(tip / total_bill)"
    valueFormat=".0%"
  />
</Columns>

<Columns cols={3}> just lays its children out side by side. Each <Metric> names a table and a value to compute (those are SQL aggregates; much more on SQL in step 2); the optional valueFormat formats the result, here as dollars and percentages. Those very tags, written into this page, render as live numbers, computed right now from db.tips:

Parties
Avg bill
Avg tip

From here on the tutorial shows each component twice: first the tag you write, then the result it renders, right below it. The simplest tag is <Table>: give it a table name with data= and it draws every row, as-is:

<Table data="db.tips" />
Connecting…

Like charts and metrics, <Table> names its source with data=. The only difference is what each does with it: a <Table> shows that source unchanged, while charts and metrics compute from it. (All of them also accept a query= with raw SQL instead; you'll use that in step 4.)

None of these tags need importing: chart, metric, and table components are available in every .mdx. Only a plain React component, like the Slider in step 4, needs the import line at the top of this file.

2. Shaping it with SQL: a view

A raw table is rarely what a chart wants. A chart wants a small, tidy result: one row per bar, the numbers already added up. A view gets you there with SQL. Mark a function with @tips.view() and return a query as a string:

@tips.view()
def tip_pct_by_day():
    return """
        SELECT day, avg(tip / total_bill) AS tip_pct
        FROM tips
        GROUP BY day
    """

Notice it's @tips.view(), not @db.view(): you're building on top of the tips table, which is why the query can just say FROM tips. The result is a new table called tip_pct_by_day. Back on the page, one tag turns it into bars. You name the table, then which column is the x-axis and which is the y:

<BarChart
  data="db.tip_pct_by_day"
  x="day"
  y="tip_pct"
  valueFormat=".0%"
  sort={["Thur", "Fri", "Sat", "Sun"]}
/>
Loading…

The sort pins the bar order: a chart draws bars in the order rows arrive, so you name the order you want rather than relying on the query.

This SQL runs in the browser. That's why a view is instant, and why it's the right tool for slicing, filtering, and adding things up.

3. Shaping it with pandas: a derive

Sometimes SQL is the awkward way to say what you mean. When Python would read more clearly (a reshape, a rolling average, a scikit-learn fit) use a derive instead. It does the same job as a view, but in pandas: you receive the parent table as a DataFrame and return a new one.

@tips.derive()
def tip_pct_by_size(df):
    df = df.copy()
    df["tip_pct"] = df["tip"] / df["total_bill"]
    return df.groupby("size", as_index=False)["tip_pct"].mean()

The page side is exactly like step 2: the same <BarChart> tag, just pointed at the new table and its columns:

<BarChart data="db.tip_pct_by_size" x="size" y="tip_pct" valueFormat=".0%" />
Loading…

View or derive? Two languages for one idea: carve a table into what a chart needs. SQL runs in the browser and is great at aggregation; pandas runs in Python and is great at reshapes and the scientific stack. Pick whichever reads more clearly. Either way, the result is fixed once the page loads.

4. Reacting to the reader: an rpc

The three blocks so far all produce a fixed result: the page loads, the tables compute once, done. But often you want the page to answer a question the reader is asking right now: recompute as they drag a slider or pick an option.

That's an RPC: a Python function the page calls with live arguments, re-run on the server every time an argument changes. Use it for the two things a view and a derive can't do: take input from a control, and run real Python that has no place in the browser.

This step ties the others together: the function that does the work, the control the reader touches, and the call that connects them.

The function. Here we fit a line to the real checks (a regression of tip on total_bill) and read it off for whatever bill the reader dials in. An RPC needs one piece of ceremony a view and derive don't. You declare the columns it returns, so the page knows the shape before the function has run:

class TipModel(Schema):
    total_bill = pa.float64()
    tip = pa.float64()
    kind = pa.string()   # 'actual tips' | 'model fit' | 'your bill'

@db.rpc()
def predict_tip(bill) -> Table[TipModel]:
    bill = float(bill)                       # the control's value arrives here
    data = tips()                            # the same rows the page sees

    # The model itself: real Python that has no place in the browser's SQL.
    slope, intercept = np.polyfit(data["total_bill"], data["tip"], 1)

    actual = pd.DataFrame({"total_bill": data["total_bill"], "tip": data["tip"], "kind": "actual tips"})
    xs = np.linspace(data["total_bill"].min(), data["total_bill"].max(), 50)
    fit = pd.DataFrame({"total_bill": xs, "tip": slope * xs + intercept, "kind": "model fit"})
    you = pd.DataFrame({"total_bill": [bill], "tip": [slope * bill + intercept], "kind": "your bill"})

    return pd.concat([actual, fit, you], ignore_index=True)

The control. Now hand the reader something to drag. The slider is wired to a piece of page state: a value the page remembers and re-renders on. You declare it once, in this file's frontmatter at the very top:

state:
  billAmount: 25

At build time n6k turns each state key into ordinary React state (literally const [billAmount, setBillAmount] = useState(25)), so anywhere in this file you get a variable holding the current value and a setter to change it. The setter's name is set + the key with its first letter capitalized (billAmountsetBillAmount). Name your state keys in camelCase and the React that reads them stays idiomatic. That's the convention across every demo.

So the whole wiring is just: read the value, and set it on change:

<Slider
  min={5}
  max={60}
  step={1}
  value={[billAmount]}
  onValueChange={(v) => setBillAmount(v[0])}
/>

The live control below adds a label around that same <Slider>:

The call. Because the RPC takes an argument, the page calls it inside a query with query=, instead of just pointing at a table with data=. We pass the slider's value straight in, and notice we can SELECT ... FROM db.predict_tip(...) as if it were an ordinary table. That's what the Table[TipModel] annotation bought us: the page already knows the function's columns, so SQL can read from it before it has even run.

This is the one place the browser/server line from step 2 is crossed, and it's worth being clear about: predict_tip(...) runs in Python on the server (that's the whole point of an RPC), its rows stream back, and the surrounding SELECT ... FROM still runs in the browser, exactly as a view does.

First the headline number, the model's expected tip for the current bill:

<Metric
  label="Predicted tip"
  query={`SELECT tip FROM db.predict_tip(${billAmount}) WHERE kind = 'your bill'`}
  valueFormat="$,.2f"
/>
Predicted tip

And the same call, plotted, the cloud of actual checks, the fitted line through them, and your bill riding along it:

<ScatterChart
  query={`SELECT * FROM db.predict_tip(${billAmount})`}
  x="total_bill"
  y="tip"
  hue="kind"
  xTitle="total bill ($)"
  yTitle="tip ($)"
  legend="bottom"
/>
Loading…

Drag the slider. Every move sends the new number to the server, predict_tip runs again, and the highlighted point slides along the fitted line. No reload, no code edit. That's the whole point of an RPC: the page is now asking Python a question live.

What you can build from here

Those four blocks (data, view, derive, rpc) are the entire framework. Everything else is a bigger version of one of them:

  • More controls, real models. The RPC demo is section 4 scaled up: two dropdowns and a slider re-fitting scikit-learn on the server every time you touch them.
  • The same live recompute, three ways. The Revenue forecast trio takes one RPC and moves where it runs (in the render loop, behind a data.py, then as a pure React frontend), so you can see the trade-offs the choice buys you.
  • Data that keeps arriving. The streaming demo is an RPC that yields results over time instead of all at once. A finite function like predict_tip above returns one DataFrame; a streaming RPC instead yields a row at a time, and each yield can be a dict, a list of dicts, or a DataFrame; n6k coerces them to the same shape.
  • A finished dashboard. Social Media Intelligence shows these same pieces themed and laid out as a real product.