Sparse Operators for Faster Model Inference

A case study of how we can use sparse operators in ONNX to improve model performance. I use VowpalWabbit in order to showcase the benefits of sparse inference. This was work done when I worked for the RL Open Source Fest program hosted by Microsoft Research NYC.

Equations

This theme supports rendering beautiful math in inline and display modes using MathJax 3 engine. You just need to surround your math expression with $$, like $$ E = mc^2 $$. If you leave it inside a paragraph, it will produce an inline expression, just like \(E = mc^2\).

To use display mode, again surround your expression with $$ and place it as a separate paragraph. Here is an example:

\[\left( \sum_{k=1}^n a_k b_k \right)^2 \leq \left( \sum_{k=1}^n a_k^2 \right) \left( \sum_{k=1}^n b_k^2 \right)\]

Note that MathJax 3 is a major re-write of MathJax that brought a significant improvement to the loading and rendering speed, which is now on par with KaTeX.


Citations

Citations are then used in the article body with the <d-cite> tag. The key attribute is a reference to the id provided in the bibliography. The key attribute can take multiple ids, separated by commas.

The citation is presented inline like this: (a number that displays more information on hover). If you have an appendix, a bibliography is automatically created and populated in it.

Distill chose a numerical inline citation style to improve readability of citation dense articles and because many of the benefits of longer citations are obviated by displaying more information on hover. However, we consider it good style to mention author last names if you discuss something at length and it fits into the flow well — the authors are human and it’s nice for them to have the community associate them with their work.


Footnotes

Just wrap the text you would like to show up in a footnote in a <d-footnote> tag. The number of the footnote will be automatically generated.This will become a hoverable footnote.


Code Blocks

Syntax highlighting is provided within <d-code> tags. An example of inline code snippets: <d-code language="html">let x = 10;</d-code>. For larger blocks of code, add a block attribute:

var x = 25; function(x) { return x * x; }

Layouts

The main text column is referred to as the body. It is the assumed layout of any direct descendants of the d-article element.

.l-body

For images you want to display a little larger, try .l-page:

.l-page

All of these have an outset variant if you want to poke out from the body text a little bit. For instance:

.l-body-outset

.l-page-outset

Occasionally you’ll want to use the full browser width. For this, use .l-screen. You can also inset the element a little from the edge of the browser by using the inset variant.

.l-screen

.l-screen-inset

The final layout is for marginalia, asides, and footnotes. It does not interrupt the normal flow of .l-body sized text except on mobile screen sizes.

.l-gutter