MDN Web Docs evolves! Lowdown on the upcoming new platform

Update, November 3: The Yari beta phase is now open, so we’ve removed the beta signup form from this post. If you want to participate in beta testing, you can find the details on our Yari beta launch explainer.

The time has come for Kuma — the platform that powers MDN Web Docs — to evolve. For quite some time now, the MDN developer team has been planning a radical platform change, and we are ready to start sharing the details of it. The question on your lips might be “What does a Kuma evolve into? A KumaMaMa?”

What? Kuma is evolving!

For those of you not so into Pokémon, the question might instead be “How exactly is MDN changing, and how does it affect MDN users and contributors”?

For general users, the answer is easy — there will be very little change to how we serve the great content you use everyday to learn and do your jobs.

For contributors, the answer is a bit more complex.

The changes in a nutshell

In short, we are updating the platform to move the content from a MySQL database to being hosted in a GitHub repository (codename: Project Yari).

Congratulations! Your Kuma evolved into Yari

The main advantages of this approach are:

  • Less developer maintenance burden: The existing (Kuma) platform is complex and hard to maintain. Adding new features is very difficult. The update will vastly simplify the platform code — we estimate that we can remove a significant chunk of the existing codebase, meaning easier maintenance and contributions.
  • Better contribution workflow: We will be using GitHub’s contribution tools and features, essentially moving MDN from a Wiki model to a pull request (PR) model. This is so much better for contribution, allowing for intelligent linting, mass edits, and inclusion of MDN docs in whatever workflows you want to add it to (you can edit MDN source files directly in your favorite code editor).
  • Better community building: At the moment, MDN content edits are published instantly, and then reverted if they are not suitable. This is really bad for community relations. With a PR model, we can review edits and provide feedback, actually having conversations with contributors, building relationships with them, and helping them learn.
  • Improved front-end architecture: The existing MDN platform has a number of front-end inconsistencies and accessibility issues, which we’ve wanted to tackle for some time. The move to a new, simplified platform gives us a perfect opportunity to fix such issues.

The exact form of the platform is yet to be finalized, and we want to involve you, the community, in helping to provide ideas and test the new contribution workflow! We will have a beta version of the new platform ready for testing on November 2, and the first release will happen on December 14.

Simplified back-end platform

We are replacing the current MDN Wiki platform with a JAMStack approach, which publishes the content managed in a GitHub repo. This has a number of advantages over the existing Wiki platform, and is something we’ve been considering for a number of years.

Before we discuss our new approach, let’s review the Wiki model so we can better understand the changes we’re making.

Current MDN Wiki platform

 

workflow diagram of the old kuma platform

It’s important to note that both content contributors (writers) and content viewers (readers) are served via the same architecture. That architecture has to accommodate both use cases, even though more than 99% of our traffic comprises document page requests from readers. Currently, when a document page is requested, the latest version of the document is read from our MySQL database, rendered into its final HTML form, and returned to the user via the CDN.

That document page is stored and served from the CDN’s cache for the next 5 minutes, so subsequent requests — as long as they’re within that 5-minute window — will be served directly by the CDN. That caching period of 5 minutes is kept deliberately short, mainly due to the fact that we need to accommodate the needs of the writers. If we only had to accommodate the needs of the readers, we could significantly increase the caching period and serve our document pages more quickly, while at the same time reducing the workload on our backend servers.

You’ll also notice that because MDN is a Wiki platform, we’re responsible for managing all of the content, and tasks like storing document revisions, displaying the revision history of a document, displaying differences between revisions, and so on. Currently, the MDN development team maintains a large chunk of code devoted to just these kinds of tasks.

New MDN platform

 

workflow diagram of the new yari platform

With the new JAMStack approach, the writers are served separately from the readers. The writers manage the document content via a GitHub repository and pull request model, while the readers are served document pages more quickly and efficiently via pre-rendered document pages served from S3 via a CDN (which will have a much longer caching period). The document content from our GitHub repository will be rendered and deployed to S3 on a daily basis.

You’ll notice, from the diagram above, that even with this new approach, we still have a Kubernetes cluster with Django-based services relying on a relational database. The important thing to remember is that this part of the system is no longer involved with the document content. Its scope has been dramatically reduced, and it now exists solely to provide APIs related to user accounts (e.g. login) and search.

This separation of concerns has multiple benefits, the most important three of which are as follows:

  • First, the document pages are served to readers in the simplest, quickest, and most efficient way possible. That’s really important, because 99% of MDN’s traffic is for readers, and worldwide performance is fundamental to the user experience.
  • Second, because we’re using GitHub to manage our document content, we can take advantage of the world-class functionality that GitHub has to offer as a content management system, and we no longer have to support the large body of code related to our current Wiki platform. It can simply be deleted.
  • Third, and maybe less obvious, is that this new approach brings more power to the platform. We can, for example, perform automated linting and testing on each content pull request, which allows us to better control quality and security.

New contribution workflow

Because MDN content is soon to be contained in a GitHub repo, the contribution workflow will change significantly. You will no longer be able to click Edit on a page, make and save a change, and have it show up nearly immediately on the page. You’ll also no longer be able to do your edits in a WYSIWYG editor.

Instead, you’ll need to use git/GitHub tooling to make changes, submit pull requests, then wait for changes to be merged, the new build to be deployed, etc. For very simple changes such as fixing typos or adding new paragraphs, this may seem like a step back — Kuma is certainly convenient for such edits, and for non-developer contributors.

However, making a simple change is arguably no more complex with Yari. You can use the GitHub UI’s edit feature to directly edit a source file and then submit a PR, meaning that you don’t have to be a git genius to contribute simple fixes.

For more complex changes, you’ll need to use the git CLI tool, or a GUI tool like GitHub Desktop, but then again git is such a ubiquitous tool in the web industry that it is safe to say that if you are interested in editing MDN, you will probably need to know git to some degree for your career or course. You could use this as a good opportunity to learn git if you don’t know it already! On top of that there is a file system structure to learn, and some new tools/commands to get used to, but nothing terribly complex.

Another possible challenge to mention is that you won’t have a WYSIWYG to instantly see what the page looks like as you add your content, and in addition you’ll be editing raw HTML, at least initially (we are talking about converting the content to markdown eventually, but that is a bit of a ways off). Again, this sounds like a step backwards, but we are providing a tool inside the repo so that you can locally build and preview the finished page to make sure it looks right before you submit your pull request.

Looking at the advantages now, consider that making MDN content available as a GitHub repo is a very powerful thing. We no longer have spam content live on the site, with us then having to revert the changes after the fact. You are also free to edit MDN content in whatever way suits you best — your favorite IDE or code editor — and you can add MDN documentation into your preferred toolchain (and write your own tools to edit your MDN editing experience). A lot of engineers have told us in the past that they’d be much happier to contribute to MDN documentation if they were able to submit pull requests, and not have to use a WYSIWYG!

We are also looking into a powerful toolset that will allow us to enhance the reviewing process, for example as part of a CI process — automatically detecting and closing spam PRs, and as mentioned earlier on, linting pages once they’ve been edited, and delivering feedback to editors.

Having MDN in a GitHub repo also offers much easier mass edits; blanket content changes have previously been very difficult.

Finally, the “time to live” should be acceptable — we are aiming to have a quick turnaround on the reviews, and the deployment process will be repeated every 24 hours. We think that your changes should be live on the site in 48 hours as a worst case scenario.

Better community building

Currently MDN is not a very lively place in terms of its community. We have a fairly active learning forum where people ask beginner coding questions and seek help with assessments, but there is not really an active place where MDN staff and volunteers get together regularly to discuss documentation needs and contributions.

Part of this is down to our contribution model. When you edit an MDN page, either your contribution is accepted and you don’t hear anything, or your contribution is reverted and you … don’t hear anything. You’ll only know either way by looking to see if your edit sticks, is counter-edited, or is reverted.

This doesn’t strike us as very friendly, and I think you’ll probably agree. When we move to a git PR model, the MDN community will be able to provide hands-on assistance in helping people to get their contributions right — offering assistance as we review their PRs (and offering automated help too, as mentioned previously) — and also thanking people for their help.

It’ll also be much easier for contributors to show how many contributions they’ve made, and we’ll be adding in-page links to allow people to file an issue on a specific page or even go straight to the source on GitHub and fix it themselves, if a problem is encountered.

In terms of finding a good place to chat about MDN content, you can join the discussion on the MDN Web Docs chat room on Matrix.

Improved front-end architecture

The old Kuma architecture has a number of front-end issues. Historically we have lacked a well-defined system that clearly describes the constraints we need to work within, and what our site features look like, and this has led to us ending up with a bloated, difficult to maintain front-end code base. Working on our current HTML and CSS is like being on a roller coaster with no guard-rails.

To be clear, this is not the fault of any one person, or any specific period in the life of the MDN project. There are many little things that have been left to fester, multiply, and rot over time.

Among the most significant problems are:

  • Accessibility: There are a number of accessibility problems with the existing architecture that really should be sorted out, but were difficult to get a handle on because of Kuma’s complexity.
  • Component inconsistency: Kuma doesn’t use a proper design system — similar items are implemented in different ways across the site, so implementing features is more difficult than it needs to be.

When we started to move forward with the back-end platform rewrite, it felt like the perfect time to again propose the idea of a design system. After many conversations leading to an acceptable compromise being reached, our design system — MDN Fiori — was born.

Front-end developer Schalk Neethling and UX designer Mustafa Al-Qinneh took a whirlwind tour through the core of MDN’s reference docs to identify components and document all the inconsistencies we are dealing with. As part of this work, we also looked for areas where we can improve the user experience, and introduce consistency through making small changes to some core underlying aspects of the overall design.

This included a defined color palette, simple, clean typography based on a well-defined type scale, consistent spacing, improved support for mobile and tablet devices, and many other small tweaks. This was never meant to be a redesign of MDN, so we had to be careful not to change too much. Instead, we played to our existing strengths and made rogue styles and markup consistent with the overall project.

Besides the visual consistency and general user experience aspects, our underlying codebase needed some serious love and attention — we decided on a complete rethink. Early on in the process it became clear that we needed a base library that was small, nimble, and minimal. Something uniquely MDN, but that could be reused wherever the core aspects of the MDN brand was needed. For this purpose we created MDN-Minimalist, a small set of core atoms that power the base styling of MDN, in a progressively enhanced manner, taking advantage of the beautiful new layout systems we have access to on the web today.

Each component that is built into Yari is styled with MDN-Minimalist, and also has its own style sheet that lives right alongside to apply further styles only when needed. This is an evolving process as we constantly rethink how to provide a great user experience while staying as close to the web platform as possible. The reason for this is two fold:

  • First, it means less code. It means less reinventing of the wheel. It means a faster, leaner, less bandwidth-hungry MDN for our end users.
  • Second, it helps address some of the accessibility issues we have begrudgingly been living with for some time, which are simply not acceptable on a modern web site. One of Mozilla’s accessibility experts, Marco Zehe, has given us a lot of input to help overcome these. We won’t fix everything in our first iteration, but our pledge to all of our users is that we will keep improving and we welcome your feedback on areas where we can improve further.

A wise person once said that the best way to ensure something is done right is to make doing the right thing the easy thing to do. As such, along with all of the work already mentioned, we are documenting our front-end codebase, design system, and pattern library in Storybook (see Storybook files inside the yari repo) with companion design work in Figma (see typography example) to ensure there is an easy, public reference for anyone who wishes to contribute to MDN from a code or design perspective. This in itself is a large project that will evolve over time. More communication about its evolution will follow.

The future of MDN localization

One important part of MDN’s content that we have talked about a lot during the planning phase is the localized content. As you probably already know, MDN offers facilities for translating the original English content and making the localizations available alongside it.

This is good in principle, but the current system has many flaws. When an English page is moved, the localizations all have to be moved separately, so pages and their localizations quite often go out of sync and get in a mess. And a bigger problem is that there is no easy way of signalling that the English version has changed to all the localizers.

General management is probably the most significant problem. You often get a wave of enthusiasm for a locale, and lots of translations done. But then after a number of months interest wanes, and no-one is left to keep the translations up to date. The localized content becomes outdated, which is often harmful to learning, becomes a maintenance time-suck, and as a result, is often considered worse than having no localizations at all.

Note that we are not saying this is true of all locales on MDN, and we are not trying to downplay the amount of work volunteers have put into creating localized content. For that, we are eternally grateful. But the fact remains that we can’t carry on like this.

We did a bunch of research, and talked to a lot of non-native-English speaking web developers about what would be useful to them. Two interesting conclusions were made:

  1. We stand to experience a significant but manageable loss of users if we remove or reduce our localization support. 8 languages cover 90% of the accept-language headers received from MDN users (en, zh, es, ja, fr, ru, pt, de), while 14 languages cover 95% of the accept-languages (en, zh, es, ja, fr, ru, pt, de, ko, zh-TW, pl, it, nl, tr). We predict that we would expect to lose at most 19% of our traffic if we dropped L10n entirely.
  2. Machine translations are an acceptable solution in most cases, if not a perfect one. We looked at the quality of translations provided by automated solutions such as Google Translate and got some community members to compare these translations to manual translations. The machine translations were imperfect, and sometimes hard to understand, but many people commented that a non-perfect language that is up-to-date is better than a perfect language that is out-of-date. We appreciate that some languages (such as CJK languages) fare less well than others with automated translations.

So what did we decide? With the initial release of the new platform, we are planning to include all translations of all of the current documents, but in a frozen state. Translations will exist in their own mdn/translated-content repository, to which we will not accept any pull requests. The translations will be shown with a special header that says “This is an archived translation. No more edits are being accepted.” This is a temporary stage until we figure out the next step.

Note: In addition, the text of the UI components and header menu will be in English only, going forward. They will not be translated, at least not initially.

After the initial release, we want to work with you, the community, to figure out the best course of action to move forward with for translations. We would ideally rather not lose localized content on MDN, but we need to fix the technical problems of the past, manage it better, and ensure that the content stays up-to-date.

We will be planning the next phase of MDN localization with the following guiding principles:

  • We should never have outdated localized content on MDN.
  • Manually localizing all MDN content in a huge range of locales seems infeasible, so we should drop that approach.
  • Losing ~20% of traffic is something we should avoid, if possible.

We are making no promises about deliverables or time frames yet, but we have started to think along these lines:

  • Cut down the number of locales we are handling to the top 14 locales that give us 95% of our recorded accept-language headers.
  • Initially include non-editable Machine Learning-based automated translations of the “tier-1” MDN content pages (i.e. a set of the most important MDN content that excludes the vast long tail of articles that get no, or nearly no views). Ideally we’d like to use the existing manual translations to train the Machine Learning system, hopefully getting better results. This is likely to be the first thing we’ll work on in 2021.
  • Regularly update the automated translations as the English content changes, keeping them up-to-date.
  • Start to offer a system whereby we allow community members to improve the automated translations with manual edits. This would require the community to ensure that articles are kept up-to-date with the English versions as they are updated.

Acknowledgements

I’d like to thank my colleagues Schalk Neethling, Ryan Johnson, Peter Bengtsson, Rina Tambo Jensen, Hermina Condei, Melissa Thermidor, and anyone else I’ve forgotten who helped me polish this article with bits of content, feedback, reviews, edits, and more.

About Chris Mills

Chris Mills is a senior tech writer at Mozilla, where he writes docs and demos about open web apps, HTML/CSS/JavaScript, A11y, WebAssembly, and more. He loves tinkering around with web technologies, and gives occasional tech talks at conferences and universities. He used to work for Opera and W3C, and enjoys playing heavy metal drums and drinking good beer. He lives near Manchester, UK, with his good lady and three beautiful children.

More articles by Chris Mills…


46 comments

  1. Lawrence

    So Mozilla is just giving up and hosting everything on Amazon and Microsoft clouds? Great work everyone! Apparently they won.

    October 29th, 2020 at 09:31

    Reply

    1. Chris Mills

      FWIW, we’ve used AWS for a number of years. Did you have any feedback on the actual new platform direction?

      October 29th, 2020 at 09:52

      Reply

    2. GU

      That’s what you took from the whole article?

      October 29th, 2020 at 21:07

      Reply

  2. Gabriel Florit

    Chris, I tried to fill out the form but it tells me “This form can only be viewed by users in the owner’s organization.”

    October 29th, 2020 at 10:11

    Reply

    1. Chris Mills

      Ooops! Think I’ve fixed it now. Can you try it again?

      October 29th, 2020 at 10:32

      Reply

  3. 0xjac

    Was GitLab considered (either gitlab.com, or self-hosted) before going with GitHub?

    More than the cloud on which the platform is hosted, using private software like GitHub when there are great open-source alternatives seems to go against your principles, especial numbers 2 & 7.

    October 29th, 2020 at 10:15

    Reply

    1. chris jones

      I feel like that’s a good point why github and not gitlab or gitea???

      Shame the author is ignoring this one

      November 1st, 2020 at 15:10

      Reply

      1. Chris Mills

        I wasn’t ignoring it, merely thinking carefully about how to reply.

        The only real answer here is that we have a very small development team, a tight timescale, and a lot of previous investment in using GitHub for our work (you’ll see that all of our existing platform, demo repos, macro system, browser compat data etc. is based in GitHub). To uproot everything and move to a different platform is simply not a viable option for us right now.

        November 2nd, 2020 at 02:12

        Reply

  4. Baris

    Thank you for focusing accessibility and localization.
    It seems to me we are in the final years of worrying about l10n, NLP and ML translations are improving at a great speed.
    100% agree with the notion “non-perfect language that is up-to-date is better”.

    October 29th, 2020 at 10:27

    Reply

    1. Chris Mills

      Thanks for the comment Baris.

      October 30th, 2020 at 10:18

      Reply

  5. yeah

    Casual reminder that GitHub works with ICE to enable human rights abuses.

    October 29th, 2020 at 12:06

    Reply

  6. Adam Williams

    I hate to say it, since I’m generally supportive of MDN initiatives … but this is a real shame. You’re throwing away the benefits of a incredible wiki-based platform (where changes are pretty much instantaneous and any of us can make a difference!), the (actually pretty decent) WYSIWYG editor and an overall frictionless editing experience and replacing it with what, some (not even Markdown) document files in a Git repository.

    You’ve massively increased the bar to contribute. And for what, some small minority of edits that are vandalism?

    October 29th, 2020 at 12:12

    Reply

    1. Chris Mills

      In the post we detail a number of reasons why we are moving to a different platform, and it goes way beyond a bit of vandalism ;-)

      I am interested to see how this will change the landscape of the contributors we get. GitHub is harder to use than the Wiki, for sure, but:

      1. What we will lose in terms of casual edits, we will probably gain in terms of more substantial contributions. For each person I hear worrying about the former, I hear someone else saying that they will be more likely to contribute if it is in git and not in a Wiki.
      2. You could use the GitHub UI edit button to make a simple change to a document, without needing to know any Git commands.
      3. Not many people will edit MDN that have no interrest at all in web development, and git is a very much ubiquitous web development tool. They will need to know it eventually.

      I think there is also some education to be done here, making sure we have clear guides on how to do certain things. And we could provide helper commands for some of the tougher stuff.

      October 30th, 2020 at 10:41

      Reply

  7. robbaweba

    Thank you for the summary of MDN changes coming in the future! I’m pretty excited to experience it first-hand.

    Regarding the new (albeit small) barrier-to-entry for contributors: one suggestion that comes to mind would be adopting Github Codespaces as a way to provide a ready-to-use development environment. That way new contributors are less likely to be deterred by the setup process. If Github Codespaces is not available, then perhaps the regular VS Code “Dev Containers” might provide a similar benefit.

    October 29th, 2020 at 14:39

    Reply

    1. Chris Mills

      Some good ideas here, thanks!

      We are certain aiming to eventally provide useful sets of plugins to make MDN editing awesome in the most popular IDEs.

      I’ve not looked into GitHub codespaces, but I will do!

      October 30th, 2020 at 10:44

      Reply

  8. iamwillshepherd

    This is dope for developers. When do you expect to have issues in the repo for new docs or docs that need attention? Would contributors need to create an issue before they submit a PR?

    For non-developers, this does seem like a step back. I was under the impression that some of the best docs are produced by technical writes. Should they need to know or care about git? Seems to me this move makes their life much more difficult while also placing an unfair burden on folks living in developing countries (or America if you’re not lucky enough to have the right hue or win geography lottery).

    I do hope to contribute, but I hope there are some plans to make it easier for folks using their phone. FWIW, this was written from a phone in a developing nation.

    October 29th, 2020 at 14:43

    Reply

    1. Chris Mills

      To give you some answers:

      1. Issues to be fixed on existing docs will continue to be submitted here: https://github.com/mdn/sprints/issues
      2. Contributors would not need to create an issue before they submit a PR.
      3. Most technical writers that I know working in this space tend to have basic git knowledge at least. But even for people that don’t, there is the GitHub UI edit button, which makes things easy.
      4. Contributing to MDN from a phone has always been challenging, and I can’t promise that getting this worked out is high on our priority list. But I’d love to hear you ideas.

      October 30th, 2020 at 10:49

      Reply

      1. William

        1. I’ll review your ZenHub board later this evening (already pulled up in separate tab).
        2. This is great!
        3. Most technical writers I know in America are very familiar with git, but I’ve learned this is something I took for granted. I do understand you nor I can do anything about it, but I did think it was worth sharing.
        4. Maybe this would be better done in an issue. This would likely not serve me well given changes happen through git now (FWIW, I prefer git, but I still understand it’s a barrier to entry). Should I create issue in mdn/sprints?

        November 20th, 2020 at 05:17

        Reply

  9. Aslan

    No Gitlab?

    October 29th, 2020 at 15:12

    Reply

    1. Peter Bengtsson

      See other replies on the same subject.

      November 3rd, 2020 at 17:14

      Reply

  10. Philip Whitehouse

    Will attribution of existing contributions be retained across this migration?

    October 29th, 2020 at 18:19

    Reply

    1. Chris Mills

      We are hoping to preserve this to some degree at least. AIUI, we don’t need to keep the entire history or every edit, but we do need to list the historical contribitors somewhere. I’d be up for working out some kind of a half way solution where we list all the contributors to each page, and the number of edits each person made.

      What would be acceptable here, in your mind?

      October 30th, 2020 at 10:52

      Reply

  11. Duncan Lock

    It seems like switching to a lightweigh markup language like Asciidoc (or Markdown if you have to) sooner rather than later, would help reduce both translation and contribution friction quite a bit – as well as probably making ML translation simpler?

    October 29th, 2020 at 20:34

    Reply

    1. Chris Mills

      I agree. We are hoping to get to markdown at some point, as it is so ubiquitous on the web, but we may look at some kind of MDN flavor markdown, with a few extensions to make life easier

      October 30th, 2020 at 10:53

      Reply

  12. voracity

    I guess I have the same reservations about moving to private platforms as some of the other comments. I also find it perhaps a little unfortunate that under “Simplified back-end platform”, the new platform is actually more complex — it’s simpler from Mozilla’s perspective because of the outsourcing to 3rd parties. And I’m not trying to be critical, just clear with words. (The tech industry is getting increasingly problematic, and we need more Mozilla, so if this change helps, then it helps.)

    “We will be using GitHub’s contribution tools and features, essentially moving MDN from a Wiki model to a pull request (PR) model. This is so much better for contribution, allowing for intelligent linting, mass edits, and inclusion of MDN docs in whatever workflows you want to add it to (you can edit MDN source files directly in your favorite code editor).”

    I suspect Wikipedia (and StackOverflow, actually) succeeded because it *didn’t* have a traditional review system. Traditional review systems are slow, overly conservative, seriously biased and inflexible. The incentive to fix an incorrect public claim is *much* greater than the incentive to check an effectively private review queue. You make it sound as if there is no trade-off.

    MDN is easily the most valuable and reliable web developer resource by a large margin (not counting Q/A resources like SO) and I wish it well on the new platform.

    October 30th, 2020 at 05:48

    Reply

    1. Chris Mills

      Thanks for your comments. They are measured, fair, and good food for thought. When we say simplified, we mean that the previous platform is very difficult to maintain, and we feel that the proposed changes are necessary to allow us to make good updates in the future, and get contributors on board.

      The review queue will be completely public , and we are hoping to get a solid, stable community on board.

      October 30th, 2020 at 10:56

      Reply

  13. Rami Yushuvaev

    Awesome news! It’s going to be easier to contribute with GIT and the GitHub web interface. But you should prepare the have thousands and maybe tens of thousands issues and PRs.

    Chris, are you going to import the docs again on November 2nd? Where should I contribute? and when should I stop using the current platform?

    October 30th, 2020 at 08:56

    Reply

    1. Chris Mills

      Thanks Rami! We are certainly going to need a robust process for managing the content reviews, yes. We currently get about 300-400 per day in the wiki model, so it will be interesting to see how that changes.

      Yes, we will import the docs so they are up-to-date in the content repo when the beta launches. When the launch happens, people can feel free to keep editing the wiki, but we’ll also expect people to test the new content repo out. Changes merged there will also be replicated in the wiki, so they don’t get out of sync.

      October 30th, 2020 at 10:59

      Reply

  14. Guillaume Grossetie

    Awesome news!
    I’m really excited about this change and I think it’s a step in the right direction.

    > we are talking about converting the content to markdown eventually, but that is a bit of a ways off

    As you said, it’s probably too soon to discuss this but I would suggest to consider AsciiDoc which has an extensible syntax. I’m pretty sure it could fit MDN specific needs :)

    October 30th, 2020 at 11:08

    Reply

    1. Peter Bengtsson

      (Yari developer here)
      Thanks for the suggestion!
      One of compelling reasons for Markdown is that there are mature tooling that works well with what we have to migrate. I.e. our macro-filled HTML.
      It’s quite possibly that AsciiDoc is as good or even better but we have take ALL variables into account, especially tooling which has many tentacles.

      November 2nd, 2020 at 05:27

      Reply

  15. azu

    > We appreciate that some languages (such as CJK languages) fare less well than others with automated translations.

    In my experince, machine translation cause errors in some langualages.
    For example, Microsoft’s documentation have used machine translation and it causes many errors.

    https://github.com/dotnet/docs.ja-jp/issues/118
    https://stackoverflow.com/questions/5274463/how-to-set-msdn-to-be-always-in-english

    :memo: Microsoft’s documentation accept a feedback for fixing these issues.

    If search engine results like Google show tranlated MDN that is machine translated , I will feel that it is spam-like site.
    Similar problems was caused in stackoverflow.

    https://ja.meta.stackoverflow.com/questions/2905/stack-overflow%E3%81%AE%E8%8B%B1%E8%AA%9E%E3%81%8B%E3%82%89%E6%97%A5%E6%9C%AC%E8%AA%9E%E3%81%AB%E6%A9%9F%E6%A2%B0%E7%BF%BB%E8%A8%B3%E3%81%95%E3%82%8C%E3%81%9F%E3%82%B3%E3%83%B3%E3%83%86%E3%83%B3%E3%83%84%E3%81%AE%E3%82%B5%E3%82%A4%E3%83%88%E3%81%AB%E3%81%A4%E3%81%84%E3%81%A6%E3%81%A9%E3%81%86%E6%80%9D%E3%81%84%E3%81%BE%E3%81%99%E3%81%8B
    https://ru.meta.stackoverflow.com/questions/8371/%d0%9d%d1%83%d0%b6%d0%bd%d1%8b-%d0%bb%d0%b8-%d0%bf%d0%b5%d1%80%d0%b5%d0%b2%d0%be%d0%b4%d1%8b-%d0%bd%d0%b0-stack-overflow-%d0%95%d1%81%d0%bb%d0%b8-%d0%b4%d0%b0-%d0%ba%d0%b0%d0%ba%d0%be%d0%b9-%d0%b2%d1%8b-%d0%b2%d0%b8%d0%b4%d0%b8%d1%82%d0%b5-%d1%81%d0%b8%d1%81%d1%82%d0%b5%d0%bc%d1%83

    I feel that non-editable Machine Learning-based automated translations is harmful.
    So, I wish that the user can choose original document in english without accept-language headers.

    If you have choosed english, mdn should redirect to english page next time. (This state should be stored in storage like cookie)
    Probably, this user not want to see machine translated page.

    October 30th, 2020 at 20:23

    Reply

  16. Outvi V

    MDN has been a useful resource for web developers, and we are happy to see the improvement on this. GitHub is a good place for developers to contribute and get recognized, so moving the editing process to GitHub might ease the editing process (especially after it moves to a GitHub-Web-UI-previewable format, e.g. Markdown), and encourage users to contribute.

    What is problematic for the readers is the availability of MDN. As we all see, GitHub is a company in the US and blocks developers in certain areas if the US government asks them to, and thus is by no means “open and accessible” as described in the Manifesto. Will MDN, as a 501(c)(3) in the US, be forced to cut the access of some developers according to the goverment order? Mozilla says it’s “non-compromised”, but depending on a infra like GitHub is obviously a step to “non-non-compromised”.

    October 30th, 2020 at 20:41

    Reply

    1. Peter Bengtsson

      > What is problematic for the readers is the availability of MDN.

      Readers won’t be using GitHub and if GitHub was to go down, it wouldn’t affect readers at all. Where they would notice is if they scroll to the bottom and click the “Report a problem…” link which will require that the reporter has a GitHub account (to file a new issue).

      In terms of READING MDN, and its availability, is backed by AWS.

      November 2nd, 2020 at 06:45

      Reply

  17. Jesper

    As a non-native English speaker who would theoretically appreciate localized text but can nonetheless read English, please:

    Plainly and unmistakably mark machine translations as such. If possible, allow showing paragraph-by-paragraph, side-by-side comparisons. Allow clicking one button to set a preference cookie or similar to switch to a mode where you will always see the original version of the text – and allow reverting it just as quickly.

    If the meaning of the shift from manual translation towards machine translation is to value and maintain accuracy and relevancy, the capacity of machine translation as the de facto single translator to be accurate and relevant is a gnarly issue.

    In my experience across MDN-like sites that have tried them, machine translations are serviceable *at best*, with a large and constant risk of misunderstandings, including loss of nuance and subtlety, but also including outright malinformation where syntax, commands or intent are completely distorted. At worst, those sites would also insist on automatically redirecting from the English version to the poorer quality machine translation. Even if the capabilities were near perfect and trained especially for the domain at hand, it would still come with undesirable aspects, and it is statistically infeasible for machine translations to maintain an acceptable level of quality across a body of work the size and breadth of MDN.

    That said, the point about allowing the material to be accessible to the majority of people in the world who simply are not fluent enough in the original language to meaningfully absorb the information is undeniable and important. MDN is the de facto authority on its subjects and has to anticipate the needs of many audiences. I hope the way forward is decided in careful dialogue with speakers of many languages and in a way that allows everyone’s preferences to be met.

    October 30th, 2020 at 22:16

    Reply

    1. Peter Bengtsson

      (Yari developer here)
      Thanks for the suggestion about “marking” the machine translated text explicitly. I think that’s a great suggestion. If a user used Google Translate themselves, on an English page, they wouldn’t need this because they’re very aware what they just did 1 minute ago.

      I think the state of translations in MDN would be extremely different if MDN was a blog and each article was a published-in-time thing that generally doesn’t organically mutate. But it adds an enormous amount of complexity if the original keeps getting changed.

      We have done a lot of thinking and we’re still thinking about all the issues around localization. It’s not easy. But first we have to focus on launching the new platform. Please stay with us and help out.

      November 2nd, 2020 at 05:43

      Reply

  18. Max

    If I recall it right, Mozilla just fired the MDN Web Docs team a few months ago. Will it affect the migration?

    October 31st, 2020 at 04:01

    Reply

    1. Chris Mills

      The entire team did not get laid off. The MDN developer team had a small reduction in capacity, and the MDN writers’ team had a significant reduction in capacity, but we are moving forward with a new strategy, and both the platform and content work is proceeding.

      October 31st, 2020 at 04:35

      Reply

    2. Peter Bengtsson

      (Yari developer here)
      It’s true, as Chris pointed out in his reply.
      That’s why we need more help for you. And you and you. Everyone who loves good documentation, please come and help out! Together we improve MDN.

      November 2nd, 2020 at 05:30

      Reply

      1. Peter Bengtsson

        “That’s why we need more help FROM you.”
        Not “for you”. Sorry.

        November 2nd, 2020 at 06:42

        Reply

  19. Young

    If the published articles link to the original doc files in the Github repo, it would make navigating inside and contributing to the unfamiliar repo less daunting.

    Thanks for all the work your team is doing in this challenging time.

    November 3rd, 2020 at 09:44

    Reply

    1. Chris Mills

      It is really interesting to hear you say that. We are already looking at adding a feature that will create a section at the bottom of each MDN article saying something like “spotted a problem on this page? File an issue or go and fix it yourself — here’s a link to the page source”. Is that the kind of thing you are talking about?

      November 3rd, 2020 at 10:09

      Reply

  20. Dennis

    How does your implementation differs from Microsoft one?
    They already have whole documentation hosted in github even with comments on their site, PR details and more.

    November 5th, 2020 at 11:26

    Reply

  21. Tom VanAntwerp

    This sounds really awesome! It makes a lot of sense both in reducing technical overhead and improving the quality of submissions by routing everything through a PR process.

    With earlier layoffs at Mozilla, I was worried about what could happen to MDN. But with all content in a GitHub repo, I’ll be much more confident that it remains open and available to anyone who needs it.

    November 6th, 2020 at 09:23

    Reply

  22. Zac Svoboda

    MDN is, an continues to be, the best resource and reference for web development! I love sharing it with people and hope that this change improves the platform.

    I see a lot of criticism in this discussion here and, while I think many of the comments come from a place a love, I want to encourage looking also at what’s being gained here rather than only what is lost. Utilizing existing products to lower the maintenance burden for the MDN team can be a huge win. These are talented folks and they should be given the opportunity to tackle the tough problems. Building an advanced i18n strategy will be a big draw and enhance this community greatly!

    November 8th, 2020 at 14:44

    Reply

  23. xty

    Good job !

    if every section have a translate button will helpful.

    reader can translate any section he want and should can toggle to english to contrast.

    November 11th, 2020 at 17:18

    Reply

  24. Henry

    I hope MDN adopts ISO 639-3 tags instead of 639-1 tags for the Chinese languages (i.e. following BCP-47). Supporting only “zh-CN”/”zh-TW” for Chinese on MDN means that non-Mandarin users cannot contribute documentation written in their native language.

    November 17th, 2020 at 17:38

    Reply

Post Your Comment