simonw Follow

Simon Willison simonw

Follow

3k followers · 133 following

Datasette
San Francisco, CA
https://simonwillison.net/
@simonw

Sponsors

Sponsoring

Achievements

Achievements

Highlights

Developer Program Member
5 discussions answered

Block or Report

Block or report simonw

Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse

simonw/README.md

Currently working on Datasette and associated projects. Read my blog or follow @simonw on Twitter.

Recent releases

datasette-hashed-urls 0.4 - 2022-03-24

datasette 0.61.1 - 2022-03-23

datasette-publish-vercel 0.13 - 2022-03-20

shot-scraper 0.9 - 2022-03-14

image-diff 0.2.1 - 2022-03-11

sqlite-utils 3.25.1 - 2022-03-11

django-sql-dashboard 1.0.2 - 2022-03-08

geojson-to-sqlite 1.0 - 2022-03-04

More recent releases

On my blog

Datasette 0.61: The annotated release notes - 2022-03-24

SQLite Happy Hour - a Twitter Spaces conversation about three interesting projects building on SQLite - 2022-03-23

Weeknotes: Tildes not dashes, and the big refactor - 2022-03-19

Instantly create a GitHub repository to take screenshots of a web page - 2022-03-14

Scraping web pages from the command line with shot-scraper - 2022-03-14

Weeknotes: Distracted by Playwright - 2022-03-12

More on simonwillison.net

TIL

Extracting web page content using Readability.js and shot-scraper - 2022-03-24

Exporting and editing a Twitter Spaces recording - 2022-03-23

Rewriting a repo to contain the history of just specific files - 2022-03-22

Using the GitHub Actions cache with npx and no package.json - 2022-03-22

Counting SQLite virtual machine operations - 2022-03-21

More on til.simonwillison.net

Pinned

datasette Public

An open source multi-tool for exploring and publishing data

Python 5.9k 409
sqlite-utils Public

Python CLI utility and library for manipulating SQLite databases

Python 727 61
dogsheep/github-to-sqlite Public

Save data from GitHub to a SQLite database

Python 181 28
s3-credentials Public

A tool for creating credentials for accessing S3 buckets

Python 105 7
git-history Public

Tools for analyzing Git history using SQLite

Python 92 6
shot-scraper Public

Tools for taking automated screenshots of websites

Python 390 17

5,099 contributions in the last year

Learn how we count contributions

Less More

Activity overview

Contributed to CAVaccineInventory/vial, simonw/datasette, simonw/sqlite-utils and 5 other repositories

Contribution activity

March 2022

Created 337 commits in 55 repositories

Created 15 repositories

simonw/graphql-scraper Mar 22
simonw/awesome-django-blogs Mar 18
simonw/datasette-plugin-template-repository-demo Python Mar 16
simonw/click-app-template-repository-demo Python Mar 16
simonw/python-lib-template-repository-demo Python Mar 16
simonw/datasette-hashed-urls Python Mar 15
simonw/ijson Python Mar 15
simonw/simonwillison-net-shot Mar 14
simonw/scrape-hacker-news-by-domain JavaScript Mar 14
simonw/scrape-soe-wwd Roff Mar 13
simonw/shot-scraper-template Built by Mar 13
simonw/image-diff Python Mar 11
simonw/rure-python Rust Mar 11
simonw/shot-scraper-demo Mar 9
simonw/shot-scraper Python • Built by Mar 8

Created a pull request in simonw/datasette that received 7 comments

Use dash encoding for table names and row primary keys in URLs

Refs #1439. Build dash_encode / dash_decode functions Use dash encoding for row primary keys Use dash encoding for ?_next= pagination tokens U…

+183 −55 • 7 comments

Opened 4 other pull requests in 3 repositories

simonw/datasette 2 merged

Remove hashed URL mode Mar 17
Tilde encoding Mar 15

theArjun/awesome-django-blogs 1 merged

Simon Willison on Django Mar 18

ICRAR/ijson 1 merged

Fix link to YAJL Mar 15

Reviewed 1 pull request in 1 repository

simonw/datasette 1 pull request

Tilde encoding Mar 15

Created an issue in natbat/pillarpointstewards that received 36 comments

Get hosting setup and DNS pointed at it

Going to try Fly.io with Fly PostgreSQL for this one. Classic Django PostgreSQL app deployed via GitHub Actions.

36 comments

Opened 119 other issues in 21 repositories

simonw/shot-scraper 33 closed 8 open

shot-scraper GitHub repository template implementation Mar 14
shot-scraper GitHub repository template Mar 14
Mechanism for polling for new scraping updates without restarting Chromium Mar 14
Do nothing if shots.yml contains no data Mar 13
Make output: optional in YAML Mar 13
Mechanism to accept JavaScript from a file or standard input Mar 13
Support returning output from evaluated JavaScript, including as status code Mar 13
Support local files for pdf and accessibility as well Mar 13
Idea: JavaScript macros in the YAML file Mar 13
--devtools option for opening browser devtools Mar 12
Support retina mode for screenshots Mar 12
Don't write binary to terminal by default Mar 12
shot-scraper shot --interactive Mar 12
Option for scripted authentication Mar 12
Extract examples from YAML into shell script Mar 12
Validate YAML mini-language using Pydantic Mar 12
YAML configuration for PDF shots Mar 12
More options for shot-scraper pdf Mar 12
shot-scraper accessibility -o option Mar 12
shot-scraper pdf command for taking a PDF screenshot Mar 12
Add --javascript option to shot-scraper accessibility Mar 10
shot-scraper accessibility URL command for dumping accessibility tree Mar 10
Support taking a shot that boxes multiple selectors Mar 10
Add the Promise pattern to the docs Mar 10
if the CSS selector is used and the container is taller than the viewport, it seems to cut it Mar 10
Some issues not shown.

simonw/datasette 8 open 20 closed

Mechanism for disabling faceting on large tables only Mar 24
allow_facet: False should be respected by column cog menu Mar 24
SQL queries against databases with different routes are broken Mar 23
Potential bug in numeric handling where_clause for filters Mar 22
Consider simplifying permissions for 1.0 Mar 21
Research: how much overhead does the n=1 time limit have? Mar 21
Make check_visibility() a documented API Mar 21
Remove check_permission() from BaseView Mar 21
Reconsider ensure_permissions() logic, can it be less confusing? Mar 21
Extract out check_permissions() from `BaseView Mar 21
Tweak design of /.json Mar 20
Streaming CSV spends a lot of time in table_column_details Mar 20
Refactor CSV handling code out of DataView Mar 20
Ship Datasette 0.61 Mar 20
Release 0.61 alpha Mar 20
Introduce concept of a database route, separate from its name Mar 19
Make route matched pattern groups more consistent Mar 19
Refactor URL routing to enable testing Mar 19
Document the internals that were used in datasette-hashed-urls Mar 16
Remove Hashed URL mode Mar 15
Refactor and simplify Datasette routing and views Mar 15
Revert main to version that passes tests Mar 15
Tilde encoding: use ~ instead of - for dash-encoding Mar 14
Adopt a code of conduct Mar 7
Mechanism to default a table to sorting by multiple columns Mar 7
Some issues not shown.

natbat/pillarpointstewards 3 closed 8 open

Initial homepage Mar 17
Start a basic pytest test suite Mar 17
Get continuous deployment to Fly from GitHub Actions working Mar 17
Initial local development environment Mar 17
Support user profile photos Mar 16
Google calendar subscription for users Mar 16
Accessible version of red/green poster Mar 16
Calculate shifts based on tide times Mar 16
Show upcoming shifts, allow people to assign themselves Mar 16
Admin timeline page Mar 16
Try Auth0 for accounts and login Mar 16

simonw/datasette-hashed-urls 9 closed

Deploy demo automatically when tests pass Mar 24
Live demo of new route fixes Mar 20
Maybe use link rel=preload for the redirects Mar 16
Plugin setting to configure cache TTL Mar 16
Extend the explanation in the README Mar 15
Use cache header that is compatible with Vercel Mar 15
Use hyphen instead of underscore for compatibility with old hash URL mode Mar 15
Deploy and test a live demo Mar 15
Initial design Mar 15

simonw/datasette.io 2 open 4 closed

Use shot-scraper to create tutorial screenshots Mar 14
Search page is broken - 500/404 Mar 12
Speed up homepage with faster YouTube embed Mar 8
Add code of conduct Mar 7
Faceted tag search for plugins and tools Mar 4
Adjust heading levels in release notes to fit Mar 3

simonw/image-diff 1 open 3 closed

simonw/sqlite-utils 2 closed 1 open

I forgot to include the changelog in the 3.25.1 release Mar 11
Display autodoc type information more legibly Mar 11
Optional Pandas integration Mar 5

simonw/help-scraper 1 open 1 closed

Don't record SQLite if only a version bump Mar 23
Add sqlite3 CLI help Mar 19

simonw/datasette-publish-vercel 2 closed

simonw/scrape-hacker-news-by-domain 2 closed

Days ago being recorded as num comments Mar 14
Tracking minutes/hours ago as number of comments Mar 14

ryzokuken/draft-ryzokuken-datetime-extended 1 open

Clarify what should happen if the offset and timezone disagree Mar 23

simonw/datasette-ripgrep 1 closed

Fix tests for latest Datasette Mar 22

simonw/parlgov-datasette 1 closed

Error fetching database Mar 22

simonw/til 1 closed

Add borders for tables in TILs Mar 21

simonw/package-stats 1 closed

Grab stats by system (Linux/Mac/Windows) too Mar 20

actions/cache 1 closed

Possible to use cache with npx without a package.json? Mar 19

simonw/python-lib-template-repository 1 closed

Redesign templates to have .github/workflows not generated by cookiecutter Mar 16

simonw/datasette-cluster-map 1 open

Pick up more name pairs, e.g. long,lat Mar 14

simonw/museums 1 closed

Twitter cards aren't displaying Mar 13

simonw/shot-scraper-demo 1 closed

Remove "5 hours ago" from owlsnearme.com demo to avoid tiny diffs Mar 12

simonw/datasette-psutil 1 closed

Move CI to GitHub Actions Mar 6

Started 1 discussion in 1 repository

github/feedback

[Projects beta] Ability to sort view by when each item was last modified Mar 23

98 contributions in private repositories Mar 1 – Mar 24