Wikimedia blog

News from the Wikimedia Foundation and about the Wikimedia movement

Technology

News and information from the Wikimedia Foundation’s Technology department (RSS feed).

Wikimedia engineering August 2013 report

Major news in August include:

Note: We’re also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

(more…)

Language support at Wikimania 2013 in Hong Kong

With participants from more than 80 countries, Wikimania 2013 was a great opportunity for the Language Engineering team to meet with the users from very different language communities. We shared ideas about language tools with users from around the world, and the fact that Wikimania was held in Hong Kong this year was an opportunity to specifically discuss the support of our current and future tools for the Chinese language.

Extending language settings

During the developer days which preceded the conference, we discussed how the Universal Language Selector (ULS) could support languages with multiple variants and ordering methods:

  • Ordering and grouping. When ordering and grouping items in lists (such as pages in a category), different languages have different ordering rules. The problem arises when languages provide more than one way of ordering elements. In the case of Chinese, dozens of indexing schemes exist based on different criteria such as the number of strokes or the phonetic transcription to Latin (as is done for Pinyin).
  • Variant selection. Chinese comprises many regional language varieties. In order to offer users a local experience with minimal duplication, the Chinese Wikipedia allows to annotate variant differences on articles so that users can get the content adapted to their local variant.

Thanks to our conversations with Wikipedia editor and volunteer developer Liangent, we now better understand the context and the implications of those problems for the case of Chinese, which was key to informing our design process. By understanding the possible scenarios and the frequency of use of these features, we could better decide how prominently to present them to users. With this information, we extended the designs of the ULS to include both ordering and variant selection options, and could provide initial technical guidance on how to extend the modular ULS architecture to support the above features.

Extending the designs of the ULS to add language variant and sorting scheme selection (only when languages have more than one option for those).

 

Wikimedia projects support more than 300 languages with very different needs. As illustrated above, close collaboration with volunteers from the different language communities becomes essential to guarantee that all languages are properly supported. Please contact the Language Engineering team if you find that any particular aspect of your language is not properly supported in Wikimedia projects.

(more…)

Notifications launch on mobile

Screenshot showing new notifications icon in corner

After many weeks of beta testing, the mobile development team is happy to announce this week’s release of notifications on the mobile site. Logged in users can now receive notifications on the mobile versions of Wikimedia sites just like they can on the desktop sites. This will allow mobile users to stay up-to-date on events and activities that affect them on Wikipedia and other Wikimedia projects.

Currently supported notifications include:

  • Talk page messages: when a message is left on your user talk page;
  • Mentions: when your user name is mentioned on a talk page;
  • Page reviews: when a page you created is reviewed;
  • Page links: when a page you created is linked;
  • Edit reverts: when your edits are undone or rolled back;
  • Thanks: when someone thanks you for your edit;
  • User rights: when your user rights change;
  • Welcome: when you create a new account;
  • Getting started: easy ways for new users to start editing.

To try out notifications on mobile, log in to one of the mobile websites, such as en.m.wikipedia.org, and look for the icon in the top right corner of your screen (see screenshot). If you have new notifications, a red badge will be displayed on the icon. Clicking on the icon will take you to the notifications archive page. We are currently working on a new overlay interface for notifications on mobile that will allow you to read your new notifications without leaving the page you are on. Expect to the see the new interface rolled out some time in the next few weeks.

Screenshot showing notifications

The release of notifications on mobile is part of a gradual process of bringing feature parity to the mobile websites. We want both readers and editors to feel comfortable using the mobile sites and able to accomplish most, if not all, of the same tasks they typically perform on the desktop sites. In the future, expect to see better support for talk pages, page history, and diff views, as well as further improvements to our editing and uploading interfaces on mobile. As always, we welcome your feedback and comments. Tell us what you think on Twitter @WikimediaMobile or on IRC in the #wikimedia-mobile channel.

Note that mobile notifications currently only work on sites that have the Echo extension enabled (en.wikipedia, fr.wikipedia, hu.wikipedia, pl.wikipedia, pt.wikipedia, sv.wikipedia, and Meta-wiki). Echo should be deployed to all language Wikipedias in the near future.

Ryan Kaldari
Software Engineer

Team Wikimedia Security has a spot for you

On a Friday evening in November of 2012, an errant line of code was found in MediaWiki, the software supporting Wikimedia sites and thousands of other wikis.

This potentially affected all of the sites in the world’s fifth most visited web property, including Wikipedia, Wikimedia Commons, Wiktionary, etc. It was a fixable security problem, but only if Wikimedia’s engineers were aware of the flaw.

They weren’t.

At another website, this would be the type of problem that could generate thousands of disgruntled users. Or worse.

But at the Wikimedia Foundation, even outside normal office hours, a problem like this is flagged as a bug and listed as an issue almost immediately. And not always by developers who work for the Foundation.

In this case, Wikipedia User:PleaseStand spotted the flaw and filed a report, which then went through the chain of communication to the security team. The bug was fixed before any noticeable damage was done.

Security issues are reported as security bugs in Bugzilla, or by simply sending emails in to [email protected]. And those two avenues are used constantly.

Roan Kattouw

This is of course every website administrator’s dream – a Good Samaritan user quietly and diligently pointing out security flaws. But that goodwill doesn’t exist just anywhere.

“We usually don’t directly reach out to volunteers and ask them to actively look for security issues,” says Roan Kattouw, a Senior Software Engineer at the Wikimedia Foundation and a lead developer on MediaWiki. “Usually they approach us because they find an issue.”

Kattouw knows the process well. He started as a volunteer developer in the Wikimedia movement before joining the engineering team at the Foundation. He now works with volunteer developers to patch up holes in the MediaWiki code.

(more…)

HTTPS enabled by default for logged-in users on Wikimedia sites

Today, August 28, the Wikimedia Foundation is making a change to the software that powers the Wikimedia projects: By default, all logged-in users will now be using HTTPS to access Wikimedia sites. What this does is encrypt the connection between the Wikimedia servers and the user’s browser so that the information sent between the two is not readable by anyone else. This is in response to the recent concerns over the privacy and security of our user community, and we explained the rationale for this change in our post about the future of HTTPS at Wikimedia.

What this means for you

How this works is simple: If a user wants to log in, they will be redirected to use HTTPS for the login, thus keeping their username and password secure. After they are logged in, they stay on the HTTPS version of the Wikimedia site they are using.

Excluded Countries

Some users live in areas where HTTPS is not an easy option, most times because of explicit blocking by a government. At the request of these communities, we have made an explicit exclusion for users from those affected countries. Simply put, users from China and Iran will not be required to use HTTPS for logging in, nor for viewing any Wikimedia project site.

Disabling

Are you having a slow or unreliable experience while browsing Wikimedia sites over HTTPS? Then you can turn HTTPS off in your user preferences, under the “User profile” tab: Uncheck “Always use a secure connection when logged in”. You will need to log out and log in again for the preference to take effect. But remember, you will still need to log in using the secure HTTPS process.

HELP!

For further details, please see the HTTPS page on Meta-Wiki, which is available in several languages.

Are you unable to log in and edit a Wikimedia wiki after this change? Please contact the Wikimedia Foundation Operations team via any means you find comfortable, including this blog post’s comments section, on IRC in the #wikimedia-operationsconnect channel, or via the [email protected] email address.

Greg Grossmeier
Release Manager, Wikimedia Foundation

Restoring the forgotten Javanese script through Wikimedia

There are several confusing and surprising things about the Javanese language. First, a lot of people confuse it with Japanese, or with Java, a programming language. Also, with over eighty million speakers, it is one of the ten most widely spoken languages in the world, yet it is not an official language in any country or territory.

Illuminated manuscript of Babad Tanah Jawi (History of the Javanese Land) from the 19th century.

Javanese is mainly spoken in Indonesia, on the island of Java, which gave its name to a popular variety of coffee. The only official language of that country is Indonesian, but Javanese is the main spoken language in its area. It is used in business, politics and literature. In fact, its literary tradition goes back to the tenth century, when an encyclopedia-like work titled Cantaka Parwa was written in it. Another Javanese encyclopedia was published in the nineteenth century, titled Bauwarna.

This tradition is being continued today by Wikipedians who speak that language: every day they strive to improve and enhance the Javanese Wikipedia, now having over forty thousand articles. One of them is Benny Lin. In addition to writing articles and explaining to people the Wikipedia mission, Benny’s special passion is making the Javanese language usable online not just in the more prevalent Latin alphabet but also in the ancient Javanese script.

This ancient script also known as Carakan was used for over a thousand years, and numerous books have been published in it. These days there’s little book publishing in it, though it is still used in some textbooks, in some Facebook groups and in public signs. Elsewhere the Latin alphabet is used more frequently. The younger generation is starting to forget the old script and this rich heritage becomes inaccessible. Benny hopes that transcribing classical literature for Wikisource and writing modern encyclopedic articles in this script, will revive interest in it and help the Javanese people achieve greater understanding of their own culture, and make these largely unknown treasures of wisdom accessible to people of all languages and cultures.

Javanese Wikipedia article about Joko Widodo

Benny presented a talk about this at Wikimania in Hong Kong, the international gathering of Wikipedians. There he also worked with Santhosh Thottingal and myself, developers from Wikimedia’s Language Engineering team, to improve the support for the Javanese script in Wikipedia. Thanks to this work, Wikipedias in all languages can now show text in the Javanese script, and the readers don’t have to install any fonts on their computers, because the fonts are delivered using webfonts technologies. The exquisite Javanese script has many ligatures and other special features, which require the Graphite technology for displaying. As of this writing, the only web browser that supports it is Firefox, but Graphite is Free Software, and it may become supported in other browsers in the near future.

Benny also completed his work for Javanese typing tools for Wikipedia, so now the script can not only be read, but also written easily. This technology can even be used on other sites and not just Wikipedia, using the jquery.ime library.

He sees his work as part of a larger effort by many people who care about the script. There are others, who design fonts, promote the script in different venues and research its literature. Beeny saw that he could contribute by making the fonts and typing tools more accessible through Wikipedia, and he just did it.

Wikimedians believe that the sum of all knowledge must be freely shared by all humans, and this means that it must also be shared in all languages. Passionate volunteers like Benny are the people who make this happen.

Amir E. Aharoni
Software Engineer, Language Engineering team

Wikimedia’s OTRS email response system gets an upgrade

Barely a week after Wikimedia’s volunteer-driven email response team helped Wikipedia score number one in the American Consumer Satisfaction Index’s “Internet and Social Media,” survey, the software being used to make this happen got its first upgrade in four years. This happened thanks in large part to a generous donation of time and service by Martin Edenhofer’s consulting firm Znuny GmbH.

Emails to Wikipedia and the Wikimedia Foundation’s ten other free knowledge projects are processed by free software called the Open-Ticket Request System or “OTRS.” OTRS was developed in 2001 by Martin Edenhofer and has been used by the Wikimedia Foundation since volunteers began responding to emails in 2004. Until this week, the email response team used an OTRS system that had not been upgraded since 2009, running version 2.4. The sheer volume of emails Wikimedia has received  (nearly a million emails are stored from the last 9 years!) made the feasibility of an upgrade seem to be an impossible task.

Commenting on a bug that had been filed in 2010 to request a software upgrade, Martin Edenhofer himself offered his OTRS consulting company to assist with the huge task of the upgrade as a donation to the Wikimedia Foundation. Fast forward to a little over a year later and all the pieces were put in to place by the Wikimedia Foundation’s Maggie Dennis. Znuny’s Marcel Fratczak and the Wikimedia Foundation’s Jeff Green then spent several long days upgrading Wikimedia’s OTRS system to the latest version (3.2.9) and migrated the database to a new server.

Thanks to this great upgrade to the tool we use for responding to emails from the general public, the Wikimedia volunteers who process emails tirelessly behind the scenes have the ability to work much more efficiently and effectively to make sure that the high quality of personal customer service continues.

Philippe Beaudette
Director, Community Advocacy, Wikimedia Foundation

Wikimedia engineering July 2013 report

Major news in July include:

  • Giving more editors an easy-to-use editing interface (the VisualEditor) on several Wikipedias
  • Improving language support on our sites via summer interns’ projects and easier configuration options, and asking for help translating the VisualEditor interface
  • Enabling users to edit our sites from mobile devices, like phones and tablets, and announcing a future user experience bootcamp focusing on mobile editing
  • Finishing our transition from keeping source code in Subversion to storing it in Git
  • Launching a Wikipedia Zero partnership with Aircel, giving mobile subscribers in India the potential to access Wikipedia at no data cost
  • Updating the Wikimedia movement on how we intend to protect our users’ privacy with HTTPS
  • Signing a contract with longtime MediaWiki contributors to manage MediaWiki releases for the open source community
  • Explaining how we find and gather software problems and deliver the fixes to users

Note: We’re also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

(more…)

Join the Language Engineering team at Wikimania in Hong Kong next week!

Wikimania 2013 Hong Kong

Wikimania, the largest gathering of the Wikimedia world, is just around the corner. In less than a week, hundreds of volunteers will be gathering in Hong Kong to share stories and talk about grand plans for the year ahead. The Language Engineering team of the Wikimedia Foundation also plans to join in! The team has been working consistently on improving internationalization support in MediaWiki tools and in Wikimedia projects. At Wikimania 2013, the team will be present to talk and exchange ideas about these projects.

Team members will be presenting technology sessions, organizing a translation sprint and showcasing at the DevCamp. At the Translation sprint workshop, participants will get a quick introduction to using the Translate extension on Wikimedia wikis and translatewiki.net. The Translate extension provides MediaWiki with essential features needed to do translation work. It can be used to translate simple content, wiki user interfaces and system messages.

In an interesting design session, our team’s interaction designer Pau Giner will be presenting on the challenges and complexities of designing language-conscious user interfaces. In the talk titled Improving the user experience of language tools, he will look at two significant projects developed by the team: the Universal Language Selector (ULS) and Translate UX as case studies.

Team engineers Amir Aharoni and Niklas Laxström will discuss multilingual feature enhancement possibilities for handling multimedia content meta-information on Wikimedia Commons. Multilingual Wikimedia Commons – What can we do about it? To find an answer, they will touch upon the advanced features in Translate, ULS and other tools that can be used to translate images, templates, descriptions and other graphics metadata to make Commons a truly multilingual Wiki.

A technical session, MediaWiki i18n getting data-driven and world-reusable, on improvements in MediaWiki internationalization (i18n) will be presented by Santhosh Thottingal and Niklas Laxström. They will be talking about the preference of using data-driven approach in place of custom code which helps code maintainability across multiple i18n frameworks. An example is data usage from the Unicode Common Locale Data Repository (CLDR). They will be speaking about the challenges and benefits of maintaining two code bases MediaWiki JavaScript i18n extension and the jquery.i18n library for different developer audiences.

Language team members will be part of panel discussions covering WMF Agile practices and at the Ask the Developers session. Come meet us at any of these sessions and bring your toughest language software questions along!

Complete list of Language Engineering sessions.

Runa Bhattacharjee
Outreach and QA coordinator, Language Engineering, Wikimedia Foundation

The future of HTTPS on Wikimedia projects

This post is available in 2 languages: 中文 7%English 7%

English

The Wikimedia Foundation believes strongly in protecting the privacy of its readers and editors. Recent leaks of the NSA’s XKeyscore program have prompted our community members to push for the use of HTTPS by default for the Wikimedia projects. Thankfully, this is already a project that was being considered for this year’s official roadmap and it has been on our unofficial roadmap since native HTTPS was enabled.

Our current architecture cannot handle HTTPS by default, but we’ve been incrementally making changes to make it possible. Since we appear to be specifically targeted by XKeyscore, we’ll be speeding up these efforts. Here’s our current internal roadmap:

  1. Redirect to HTTPS for log-in, and keep logged-in users on HTTPS. This change is scheduled to be deployed on August 21, at 16:00 UTC. Update as of 21 August: we have delayed this change and will now deploy it on Wednesday, August 28 at 20:00 UTC/1pm PT.
  2. Expand the HTTPS infrastructure: Move the SSL terminators directly onto the frontend varnish caches, and expand the frontend caching clusters as necessitated by increased load.
  3. Put in engineering effort to more properly distribute our SSL load across the frontend caches. In our current architecture, we’re using a source hashing based load balancer to allow for SSL session resumption. We’ll switch to an SSL terminator that supports a distributed SSL cache, or we’ll add one to our current solution. Doing so will allow us to switch to a weighted round-robin load balancer and will result in a more efficient SSL cache.
  4. Starting with smaller projects, slowly soft-enable HTTPS for anonymous users by default, gradually moving toward soft-enabling it on the larger projects as well. By soft-enable we mean changing our rel=canonical links in the head section of our pages to point to the HTTPS version of pages, rather than the HTTP versions. This will cause search engines to return HTTPS results, rather than HTTP results.
  5. Consider enabling perfect forward secrecy. Enabling perfect forward secrecy is only useful if we also eliminate the threat of traffic analysis of HTTPS, which can be used to detect a user’s browsing activity, even when using HTTPS.
  6. Consider doing a hard-enable of HTTPS. By hard-enable we mean force redirecting users from HTTP pages to the HTTPS versions of those pages. A number of countries, China being the largest example, completely block HTTPS to Wikimedia projects, so doing a hard-enable of HTTPS would probably block large numbers of users from accessing our projects at all. Because of this, we feel this action would probably do more harm than good, but we’ll continue to evaluate our options here.
  7. Consider enabling HTTP Strict Transport Security (HSTS) to protect against SSL-stripping man-in-the-middle attacks. Implementing HSTS could also lead to our projects being inaccessible for large numbers of users as it forces a browser to use HTTPS. If a country blocks HTTPS, then every user in the country that received an HSTS header would effectively be blocked from the projects.

Currently we don’t have time frames associated with any change other than redirecting logged-in users to HTTPS, but we will be making time frames internally and will update this post at that point.

Until HTTPS is enabled by default, we urge privacy-conscious users to use HTTPS Everywhere or Tor [1].

Ryan Lane
Operations Engineer, Wikimedia Foundation

[1]: There are restrictions with Tor; see Wikipedia’s information on this.

(more…)