Wikidata deployment

Wikimedia engineering activity

Wikidata deployment

Regular deployments to wikidata and clients

Group:
Start:
End:
Team:	Sam Reed, Greg Grossmeier
Lead:	aude, hoo
Status:	See updates

Status

[edit status] • [add new]

2014-02-28:

Quantities were deployed to Wikidata on February 4, and data use on Wikisource on February 28. Performance improvements were also deployed to Wikidata.

Deployment Check List

make sure all relevant repositories have been branched make sure the branch to be deployed has been tested with the correct branch of core update the relevant submodule pointers on the wmf deployment branch find all mentioned of "DEPLOY" in commit messages of any changes that are new on the branch being deployed, and build a pre/post-deployment checklists from these. See also: wikitech:How to deploy Wikidata code

mw1.24wmf12

Schedule

Phase	Date	Wikis
Phase 4	Thursday, July 10, 2014	Deploy to Wikipedias
Phase 3	Tuesday, July 8, 2014 Y Done	Deploy to Wikidata, Wikivoyage, Commons, Wikisource, Wikiquote
Phase 2 Y Done	Thursday, July 3, 2014 Monday, July 7, 2014 Y Done	Deploy to testwikidata
Phase 1 Y Done	Tuesday, July 1, 2014 Thursday, July 3, 2014 Y Done	Branchday

Changes

Bug fixes

• Fixed snak type selection (e.g. novalue) sticks when canceling. bugzilla:67294
• Fixed cannot reorder snaks within a reference. bugzilla:65695
• Fixed not possible to add no-value and some-value snaks. bugzilla:66626
• Fixed not possible to change rank alone. bugzilla:66633
• Fixed High coordinate precision fires "The value does not comply with the property's definition." bugzilla:55971
• Fixed make it possible to edit globecoordinates when precision does not match predefined precisions in the frontend bugzilla:65535
• Fixed do not link deleted items - show text note instead bugzilla:65002

Other

• Improve i18n messages, make less confusing.
• Use api serialization format for internal storage, with backwards compat. that we need for Wikidata (for now).

Deployment notes

• Add setting to continue using old serialization format for backwards compat: 'internalEntitySerializerClass' => 'Wikibase\Lib\Serializers\LegacyInternalEntitySerializer' Y Done

API Changes

• BREAKING CHANGE: wblinktitles no longer returns an integer item id, but a properly prefixed one
• BREAKING CHANGE: DispatchStats API returns ISO 8601 timestamps instead of raw database strings. (gerrit:138306)

mw1.24wmf11

Schedule

Phase	Date	Wikis
Phase 4	Thursday, July 3, 2014	not deployed to Wikipedias, since its for Wikidata only
Phase 3	Tuesday, July 1, 2014 Y Done	Deploy to Wikidata (not deployed to Wikivoyage, Commons, Wikisource, Wikiquote)
Phase 2 Y Done	Thursday, June 26, 2014	Deploy to testwikidata
Phase 1 Y Done	Wednesday, July 25, 2014	Branchday

Changes

Enabling new property / entity suggester for test / wikidata.

The suggester enhances the property suggestions on Wikidata when entering new statements. For example, an item that already has "head of government", when you enter the next statement and enter "t" and "type of administrative entity" is the first suggestion instead of "tennis singles rating" or other such less irrelevant suggestion, based on correlation statistics for properties and how often they are used together on items.

The suggester has an api module and combines results in / extends the default Wikibase javascript entity suggester.

The suggester has been reviewed for security, etc. (see bugzilla:66382) and is deployed on the Wikidata test / demo system. [1]

mw1.24wmf8

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, June 12, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, June 10, 2014	Deploy to Wikidata, Wikivoyage, Commons, Wikisource, Wikiquote
Phase 2 Y Done	Thursday, June 5, 2014	Deploy to testwikidata
Phase 1 Y Done	Tuesday, June 3, 2014	Branchday

mw1.24wmf6

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, May 29, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, May 27, 2014	Deploy to Wikidata, Wikivoyage, Commons, Wikisource, Wikiquote
Phase 2 Y Done	Thursday, May 22, 2014	Deploy to testwikidata
Phase 1 Y Done	Wednesday, May 20, 2014	Branchday

Changes

Bug fixes

• bugzilla:65646 - merge items
• improved snak error handling and formatting (fixes property parser function and lua to work with #iferror when there is a mismatch between property and data value type, or 'ununserializable value'.)
• jQuery compatibility fixes (already deployed)
• prevent import of entities to avoid ID conflicts bugzilla:63228
• fix various javascript issues for editing / saving statements bugzilla:64651, bugzilla:65694, bugzilla:65693, bugzilla:65398

Other

• moved term validation into ChangeOps
• use single instance of SiteSQLStore bugzilla:64743 (should help some with performance)

mw1.24wmf4

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, May 15, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, May 13, 2014	Deploy to Wikidata, Wikivoyage, Commons, Wikisource, Wikiquote
Phase 2 Y Done	Thursday, May 8, 2014	Deploy to testwikidata
Phase 1 Y Done	Wednesday, May 7, 2014	Branchday

Changes

Bug fixes

• fix for double pencil icon on clients (bugzilla:65001)
• fix for Special:EntityData, which was giving 503 error for large items (bugzilla:60003)
• lots of bug fixes for time formatting and parsing (mostly/all already backported and deployed)

Other

• removed links in api.php to unmaintained documentation on mediawiki.org

Deployment notes

• ...

mw1.24wmf2

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, May 1, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, April 29, 2014	Deploy to Wikidata, Wikivoyage, Commons, Wikisource, Wikiquote
Phase 2 Y Done	Thursday, April 24, 2014	Deploy to testwikidata
Phase 1 Y Done	Wednesday, April 23, 2014	Branchday

Changes

Enhancements

• use backend globe coordinate and time formatters and parsers (for input in UI)
• now using i18n json format
• changes to api error reporting
• changed UI text "source" to "reference"
• improvements to backend handling of UpdateRepoOnMoveJob

Bug fixes

• intermediate solution to bug 61854 (render links for deleted items in red)
• fix edit restriction handling - bug 58474
• fix miscounting of coalesced changes in client recent changes - bug 45812
• fix entity selector, when selecting (e.g. 3rd suggestion) property value then sometimes the 1st suggestion is saved - bug 62868

Known issues

• "Precision higher than 'DAY' is not supported yet" error for dates like 1987x - bug 64558, waiting to deploy fix deployed

Deployment notes

• update cache epoch for parser cache

mw1.23-wmf21

Schedule

Phase	Date	Wikis
Phase 4	Thursday, April 10, 2014	Deploy to Wikipedias N Not done Y Done
Phase 3	Tuesday, April 8, 2014	Deploy to Wikidata, Wikivoyage, Commons, Wikisource, Wikiquote Y Done (then reverted next day to time localisation bug) bugzilla:63723
Phase 2	Thursday, April 3, 2014	Deploy to testwikidata N Not done bugzilla:63659 Y Done
Phase 1	Tuesday, April 1, 2014	Branchday Y Done

Changes

Enhancements

• use backend globe coordinate and time formatters
• now using i18n json format

Deploy to Wikiquote

Schedule

Phase	Date	Wikis
Deploy to Wikiquote	Tuesday, April 8, 2014	Enable Wikibase on Wikiquote for language links

Deployment notes

• add sites and site_identifiers tables
• populate sites table
• config change [2]

mw1.23-wmf19

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, March 27, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, March 25, 2014	Deploy to Wikidata, Wikivoyage, Commons, Wikisource
Phase 2 Y Done	Thursday, March 20, 2014	Deploy to testwikidata
Phase 1 Y Done	Tuesday, March 18, 2014	Branchday

Changes

Enhancements

• Javascript config variables (wbEntity and wbUsedEntities) are now cached in parser cache
• Reduce use of EntityContent in various places in the code

Bug fixes

• Fixed wbsearchentities api continue param bugzilla:62959
• Fixed issue of Wikibase changes not appearing in client watchlist bugzilla:62149
• Fixed inaccurate edit summary (mentioned badges for non-badge edits) in Special:SetSiteLink bugzilla:62546
• Special:DispatchStats now displays timestamps adjusted to users' timezone bugzilla:62428
• Don't use ULS heuristics in choosing languages for the "other languages / terms" box on item pages, if a user has babel boxes bugzilla:61446

Deployment notes

• Rights have been changed, probably requires changes (OAuth) [3] - change is not required immediately and can be done anytime.

mw1.23-wmf17

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, March 13, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, March 11, 2014	Deploy to Wikivoyage, Commons, Wikisource
Phase 2 Y Done	Thursday, March 6, 2014	Deploy to test.* and Wikidata
Phase 1 Y Done	Tuesday, March 4, 2014	Branchday

Changes

Enhancements

• ...

Bug fixes

• ...

Deployment notes

• ...

mw1.23-wmf15

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, February 27, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, February 25, 2014	Deploy to Wikivoyage, Commons, Wikisource
Phase 2 Y Done	Thursday, February 20, 2014	Deploy to test.* and Wikidata
Phase 1 Y Done	Tuesday, February 18, 2014	Branchday

Changes

Enhancements

• New Lua interface
• Moving formatters to backend
• JS performance improvements
• Enable data use on Wikisource

Deployment notes

needed for wikisource

• Enable data transclusion for Wikisource on the 25th.

mw1.23-wmf11/12

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, February 6, 2014	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, February 4, 2014	Deploy to Wikivoyage, Commons, Wikisource
Phase 2 Y Done	Thursday, January 30, 2014	Deploy to test.* and Wikidata
Phase 1 Y Done	Wednesday, January 21, 2014	Branchday

Changes

Enhancements

• quantities!
• performance improvements
• “in other languages” box also show up for people who have not configured a babel box bugzilla:49079

Bug fixes

• revisions of items that have been RevDeled or suppressed (by oversighters) cannot be viewed bugzilla:58027
• claim index out of bounds

Deployment notes

needed for test.wikidata

• purge the parser cache by bumping $wgCacheEpoch [4]
• make sure wikidata-shortcopyright message is handled properly [5] Y Done
  • add new data rights url / text settings [6]
• update config to load new extension-list-wikidata / old list conditionally [7]

needed for wikidata

• stop/update cron jobs to use code in new location [8]
• old client code compatible with new repo code - with cache key varying on $wmfExtendedVersionNumber, should be fine. Y Done

Not required for deployment, but should be done soon

• schema updates - check that they don't depend on deployment... i think they can be done before/after/whatever
  • https://gerrit.wikimedia.org/r/#/c/99637/ - (bug 47135) Make row IDs use BIGINT, db schema change, see repo/sql/MakeRowIDsBig.sql
  • https://gerrit.wikimedia.org/r/#/c/99660/ - (bug 45529) use composite indexes on wb_terms, db schema change, see repo/sql/UpdateTermIndexes.sql

Deploy to Wikisource

Phase	Date	Wikis
Y Done	Tuesday, January 14, 2014	Deploy to Wikisource

Deployment notes

• add sites table and populate for wikisource

mw1.23-wmf6

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, December 12, 2013	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, December 10, 2013	Deploy to Wikidata, Wikivoyage, Commons
Phase 2 Y Done	Thursday, December 5, 2013	Deploy to test.* (+enable quantity)
Phase 1 Y Done	Wednesday, December 4, 2013	Branchday

Changes

• data type in json / api output
• table of contents
• sorting
• ranks

Other

• improvements to the link item widget in the client
• using parser cache now
• improvements to page loading in backend

mw1.23-wmf5

Schedule

Phase	Date	Wikis
Phase 4 Y Done	Thursday, December 5, 2013	Deploy to Wikipedias
Phase 3 Y Done	Tuesday, December 3, 2013	Deploy to Wikidata, Wikivoyage, Commons (+quantity disabled)
Phase 2 Y Done	Thursday, November 21, 2013	Deploy to test.* (+enable quantity)
Phase 1 Y Done	Wednesday, November 20, 2013	Branchday

Bug fixes

• diff=0 works - bugzilla:49434

See also

• wikitech:Deployments - wmf calendar
• Wikidata deployment/guide - how to do things