Our Open Source Strategy
This page has been archived.
We keep an eye on on our what our government counterparts are up to, both in the U.S. and other countries. We first came across Gov.UK’s philosophy on and approach to coding in the open a couple of years ago. It caught our attention and we realized we should also articulate our open source strategy.
Use and Contribute to Open Source Projects
Since 2010, we’ve embraced and leveraged open source software to build our site search service for federal, state, and local government websites. This use of open source has allowed us to experience enormous growth over the past few years. In July 2014 alone, over 23 million searchers received results from our service — a five-fold increase since July 2010.
Our search service is now a complex system made up of many moving parts, including providing typeahead search suggestions, serving search results, fetching, indexing, and caching content, and providing analytics.
Each of these parts is compiled into our codebase and, as we use open source components for our system, we contribute back to the projects.
Code in the Open
We recently began to unravel our monolithic codebase so that we can share individual pieces of our code. To borrow the phrase from Gov.UK, we’re now coding in the open.
We recently released the code for our social media image, jobs and recalls API servers. They’re our first foray into coding in the open. The source code for these API servers is in our GitHub repo and is available for anyone to see and contribute to.
The data products for the jobs and recalls code are also open and available for anyone to consume on our Developer hub.
These three servers and their underlying data now operate outside of our core search codebase.
Following this same model, moving forward, we plan to:
- Share first — For every new feature, we’ll write the code so that anyone can make use of the code, not just us. If the public community contributes to the codebase, we’ll be able to improve this feature without taxing our developers.
- Expose APIs — We’ll expose our data products as APIs so that anyone can make use of the data, not just searchers on a government websites.
- Be our own customer — We’ll use our own public code and data just like everyone else. We’ll call our own API servers to integrate the data within our search results pages.
Make Things Open to Make Things Better
We agree with Gov.UK that “to make things open makes things better.”
We have finite resources and we don’t want to lose our focus on serving our agency customers and improving visitors’ search experience on government websites. So, we won’t be spending a lot of time to build or support a vibrant community around our code.
That said, we hope that exposing the pieces of our system will be useful to someone somewhere. We’ll continue to provide the “ingredients” of our search service so that others will be able to make use of the code and data in ways that we could never imagine.
And, We’re Not Alone
We’re not alone. Other federal agencies have embraced the approach of coding in the open and have GitHub repos. Below are a just a few of our many favorites.
- The Consumer Financial Protection Bureau has embraced open source, including serving their public data sets through qu and sharing their work at https://cfpb.github.io.
- The Food and Drug Administration recently made public FDA datasets more accessible at openFDA.
- GSA’s newly-created 18F has committed to working in the open from day 1.
- The White House was in the forefront of using open source software in the federal government with their use of Drupal. They’re continuing to lead by example by opening up the code for their We the People petitions and iOS and Android mobile apps.