This report relied on both expert comments and data gathered before, during, and after the hurricane season. Estimates of the impacts discussed on this report utilized a variety of data sets across trade and transportation sectors. These data sets including statistics on trade patterns and commodity tonnage from previous years; records of port advisory and closure status; historic vessel position reports; and storm-related characteristics such as date, location, and wind speed.

Links to the report:
CMTS Press Release at https://www.cmts.gov/posts/AWfRaeCN3JAwUKdhhD45
or
Download the PDF “The 2017 Hurricane Season: Recommendations for a Resilient Path Forward for the Marine Transportation System” at
https://www.cmts.gov/downloads/CMTS_RIAT_2017Hurricanes.pdf

Select data sources cited in the report:
Pérez, M.C., A.D. Marrero, L.A. Rivera. (2017). “External Trade Statistics Puerto Rico 2016”, Government of Puerto Rico, Office of the Governor Planning Board. http://jp.pr.gov/Portals/0/Economia/Comercio%20Exterior/External%20Trade%20Statistics/ets-2016%20(v1).pdf?ver=2017-08-23-092107-857

U.S. Department of Commerce, National Oceanic and Atmospheric Administration (2016): “Describing the Ocean Economies of the U.S. Virgin Islands and Puerto Rico”, accessed July 2018. https://coast.noaa.gov/digitalcoast/training/econ-usvi-pr.html

U.S. Department of Commerce, National Oceanic and Atmospheric Administration, National Centers for Environmental Information: “U.S. Billion-Dollar Weather and Climate Disasters: Overview” 2018. https://www.ncdc.noaa.gov/billions/

The post After the storms: Research on How to Improve Marine Transportation System Resilience after the 2017 Hurricane Season appeared first on Data.gov.

The interactive map can be found at: https://www.epa.gov/ports-initiative/best-port-wide-planning-practices-improve-air-quality#clean-air

The Ports Initiative website also includes information on methods for performing an emissions inventory, a toolkit to assist community and port collaboration efforts, information on funding opportunities, and other technical resources.

Visit the Ports Initiative main page at: https://www.epa.gov/ports-initiative
Contact the Ports Initiative at: talkaboutports@epa.gov

The post Clean Air Practices at Ports – EPA Interactive Map appeared first on Data.gov.

The post 2019 Port Performance Data from the U.S. DOT appeared first on Data.gov.

The new catalog is the culmination of many months of work in updating the behind-the-scenes functioning of the Data.gov catalog, which automatically harvests over 1000 different sources from federal, state and local open data sources to provide a comprehensive catalog of open government data. The new catalog runs an updated version of CKAN, the open source technology behind the Data.gov catalog, and should improve the process of automatically updating Data.gov with the most recent datasets.

Most users will not notice any differences in the new catalog. At first, the front page of the catalog will show a small reduction in total datasets. That reflects the deletion of outdated datasets from harvest sources that are no longer maintained, mostly at the state and local level. As we have noted in the past, the front page number of total datasets changes frequently as many harvest sources are checked every day for updates. Also, the catalog counts collections of datasets as “1” in the total number. The validation and update of all 1,098 harvest sources should result in an improved user experience with a higher degree of accuracy and fewer broken links. In the coming months, we will deploy additional features to continue to improve the quality of the Data.gov catalog.

The post Launch of the New Data.gov Catalog appeared first on Data.gov.

Due to demand, please access the data files here:
– Download: https://healthdata.gov/sites/default/files/reported_hospital_capacity_admissions_facility-level_weekly_average_timeseries_20201207.csv
– Data Dictionary: https://healthdata.gov/covid-19-reported-patient-impact-and-hospital-capacity-facility-data-dictionary

Today, the U.S. Department of Health and Human Services (HHS) published COVID-19 hospital data at the facility level for each week, going back to August 1st 2020. This information was previously aggregated by state. This new data and its data quality are paramount to the U.S. pandemic response. COVID-19 models, analyses, and predictive analytics can only be as good as the data they ingest. By sharing these new hospital COVID-19 capacity data at the facility-level with the public, the opportunity exists to improve data quality.

This data is tremendously complex and is the result of substantial ongoing efforts by hospital, state, and Federal personnel who collaborate to meet daily data reporting requirements. This ongoing collaboration has steadily improved data quality and reporting consistency in recent weeks, even as the content of hospital capacity data has become more complex.

We opted not to have perfect be the enemy of good, so these datasets will have imperfections. To continue improving the quality of data, we welcome your feedback. When more people access and use the data, we have more collective ability to identify gaps, errors, or other problems with these COVID-19 datasets. All stakeholders — local community coordinators, data scientists, data journalists, and healthcare researchers — are encouraged to download, analyze, study the datasets, and share your feedback with HealthData.gov.

We also encourage you to visit and contribute to Frequently Asked Questions (FAQs) about this hospital-level data. These living FAQs are crowdsourced from the public, so your contributions will improve these FAQs over time.

Commitment to Transparency and Open Data

By opening COVID-19 datasets, our collective goal is to accelerate scientific and public health insights and shorten the time it takes for COVID-19 information and solutions to save lives. Our HHS efforts are enabling a data-driven ecosystem for everyone. Based on public feedback and interaction with the COVID-19 datasets, this open-data ecosystem will evolve while guided by these principles: Transparency. Sharing. Privacy. Security. Community.

Today’s unprecedented pandemic demands near-real-time data sharing across government, across diverse sectors, and with the public. HHS is aggressively responding to this call-to-action by responsibly unlocking information wherever our societal benefits of data access outweigh the potential risks. This is a collaborative effort led by HHS Office of the Chief Data Officer, within the Office of the Chief Information Officer, in close coordination with the Centers for Disease Control and Prevention (CDC), local/state/tribal governments, HHS Office of the Assistant Secretary for Health (OASH), HHS Office of the Assistant Secretary for Preparedness and Response (ASPR), and the White House U.S. Digital Service.

Beyond the whole-of-government response, COVID-19 necessitates a whole-of-America response. Your public input so far has enabled HHS to identify and prioritize information of highest value. We appreciate the hard-working journalists, data scientists, academic healthcare researchers, business innovators, and Americans who are giving time, energy, and insights to help us responsibly open data related to the novel coronavirus.

The data reporting is one part of the whole-of-government response and is to ensure that every patient requiring hospitalization receives the care they need. Patients should not be discouraged from seeking hospital care based on their interpretation of the data. Hospitals have protocols in place to keep patients safe from exposure and to ensure all patients are prioritized for care. By working together across sectors, we are harnessing all capabilities and resources to unleash the power of U.S. data for the COVID-19 response.

Together, we can combat COVID-19.

Disclaimer: Please visit the CDC’s website on COVID-19 for the most up-to-date information and COVID-19 guidance.

The post COVID-19 is Complex, as is COVID-19 Open Data appeared first on Data.gov.

As called for in the Foundations for Evidence-Based Policymaking Act and delivering against milestones set out in Action 11 of the Federal Data Strategy 2020 Action Plan, this update to resources.data.gov features new ways to browse tools and resources by category and keyword, introduces a user-driven taxonomy and information architecture, and adds new case studies, tools, and policy references to the repository. The update is the work of the Data Federation Team from GSA’s 18F, through support from the 10x program. You can learn more about this process in the 10x’s blog post on its work.

Find the following on resources.data.gov:   

• Data management & governance: Playbooks, guidance, templates, and other resources to support the implementation of policy, the creation of data governance structures, and the day-to-day work of data management in the federal government

• Data tools: Software tools and complementary resources to support the hands-on work of data practitioners

• Data standards: Standards, schemas, and related resources — this category will continue to be built out in response to Federal Data Strategy Action 20, which calls for a data standards repository to accelerate the creation and adoption of data standards across agencies

• Skills development: Guides, tutorials, and personnel-related resources to build and develop a data-savvy workforce

• Policy: Legislation, official memos, regulations, and mandates related to the governance, management, and use of data in the federal government

• Case studies & examples: Articles, use cases, and proof points describing projects undertaken by data managers and data practitioners across the federal government

This relaunch aims to capture a lot of existing work in a more discoverable and user friendly way, but the real goal is to create a lasting foundation for much more work to come. The Federal Data Strategy 2020 Action Plan calls out resources.data.gov as the place to find a number of deliverables that are still in development as well as continued stakeholder engagement to help inform and evolve the site to better serve users. Contributions to resources.data.gov are welcome through its repository on GitHub.

The post Serving the Needs of Data Practitioners with a New resources.data.gov appeared first on Data.gov.

Coastal Inundation in Your Community
Coastal communities are becoming increasingly vulnerable to the risk of damage and danger from flooding. Your challenge is to create tools and provide information so communities can prepare for coastal inundation. Through the use of data, visualizations, citizen engagement, and simulations, you can help people understand their exposure to coastal inundation hazards and their increased vulnerability due to population increase and sea level rise.

Space Apps is a two-day event (April 12-13, 2014) that has about 40 challenges organized around Earth and space themes. Teams of technologists, scientists, designers, artists, educators entrepreneurs, developers and students across the globe collaborate and engage with publicly available data to design innovative solutions for global challenges. Visit the resources tab to find datasets, web services, and tools that will help you compete in the Space Apps Challenge on the coastal impacts of climate change.

The post NASA and NOAA Launch Coastal Flooding Innovation Challenge appeared first on Data.gov.

The challenges will use the Open NASA Earth Exchange. OpenNEX is a data, supercomputing and knowledge platform where users can share modeling and analysis codes, scientific results, knowledge and expertise to solve big data challenges in the Earth sciences. A component of the NASA Earth Exchange, OpenNEX provides users a large collection of climate and Earth science satellite data sets, including global land surface images, vegetation conditions, climate observations and climate projections.

For more information, see the NASA OpenNEX Press Release and the OpenNEX site.

The post NASA Launches Earth Science Challenges with OpenNEX Cloud Data appeared first on Data.gov.

The post Launch of Food Resilience Theme appeared first on Data.gov.

The post Water Resources Resilience appeared first on Data.gov.

The post Electricity Information Agency’s Energy Disruptions Tool appeared first on Data.gov.

Analyzing data on the older adult population in the time periods before, during, and after the pandemic is an important step to gaining a better understanding of both the health of this age group as well the immediate and long-term impacts of the pandemic. The Older Adults Health Data Collection contains datasets that capture outcomes directly related to COVID-19 as well as others that do not, and the number of COVID-19 related datasets in the collection is anticipated to grow over time. Datasets without COVID-19-specific data elements still provide important context on the health of older adults before the pandemic and are likely to incorporate COVID-19 data elements in their data collection procedures in the months and years to come.

Demographic trends underscore just how vital it will be to understand this growing population group. By 2030, one in five Americans will be 65 years and older. Over the next 40 years, the number of Americans 65 years and older will almost double.¹ COVID-19 has also significantly impacted this age group, as underscored by evidence that older adults have experienced the highest hospitalization and morbidity rates from the pandemic.² This is partly the result of older adults having high rates of chronic diseases, such as heart disease, type 2 diabetes, arthritis, and cancer, which are the Nation’s leading drivers of illness, disability, deaths, and health care costs. 

Projected number of older adults in the United States1	Coronavirus disease 2019 (COVID-19)–associated hospitalization rates2

This new resource aligns with the intent of the Foundations for Evidence-Based Policymaking Act of 2018 and directives in the Federal Data Strategy 2020 Action Plan, both of which encourage increased public access to Government data, collaboration with non-Government entities, interagency collaboration, and protection of data security and confidentiality.  This data collection also supports the efforts of the Federal Interagency Forum on Aging Related Statistics, which aims to improve the quality and utility of data on the aging population.  Two of the Aging Forum’s goals include encouraging cross-national research on the aging population and promoting communication among data producers, researchers, and public policymakers. Additional non-health and non-Federal datasets related to older adults are available at agingstats.gov.

As the first national open data site, Data.gov is the ideal platform to provide a wide variety of stakeholders with access to the Older Adults Health Data Collection. Data in this collection has been made available in open formats while ensuring privacy and security, consistent with all applicable laws, regulations, and policies governing data use, disclosure, and sharing. By reducing the time spent searching for data on the health of older adults from hours to seconds, the Older Adults Health Data Collection serves as a helpful resource to put Federal data to work on behalf of Americans.

To access the Older Adults Health Data Collection, follow this link: https://catalog.data.gov/group/older-adults-health-data

Sources

1. U.S. Census Bureau, 2017 National Population Projections.
2. Garg S, Kim L, Whitaker M, et al. Hospitalization Rates and Characteristics of Patients Hospitalized with Laboratory-Confirmed Coronavirus Disease 2019 — COVID-NET, 14 States, March 1–30, 2020. MMWR Morb Mortal Wkly Rep 2020;69:458–4  64. DOI: http://dx.doi.org/10.15585/mmwr.mm6915e3

Vijeth Iyengar, PhD, is a Policy Advisor at the White House Office of Science and Technology Policy

Mark C. Bicket, MD, PhD, is a White House Fellow at the White House Office of Science and Technology Policy

The post Improving Access to Older Adult Health Data for Timely Use Amid COVID-19 and Beyond appeared first on Data.gov.

We have also added navigation to two existing sites, strategy.data.gov and resources.data.gov. Strategy.data.gov is the home of the Federal Data Strategy, with information about the strategy, the Action Plan, and progress toward the Action Plan items, as determined by the Office of Management and Budget. Resources.data.gov is an online repository of tools, best practices, and schema standards to facilitate adoption of open data practices. The site is a joint effort of OMB, the Office of Government Information Services of the National Archives, and GSA, as required by the OPEN Government Data Act.

In addition, we have conducted a content review of the site to archive outdated content. Datasets are not affected, as the Data.gov catalog is continually updated automatically by harvesting metadata maintained by the contributing agencies. Under the OPEN Government Data Act, federal agencies maintain comprehensive inventories of their data assets that are harvested into the federal data catalog, Data.gov. Progress on agency data inventories can be tracked on this dashboard. Overall progress on the Federal Data Strategy is reported on strategy.data.gov.

The post Updates to Data.gov Home Page appeared first on Data.gov.

The International Energy Portal contains EIA’s country-level energy data. Users can view and download datasets for consumption, production, trade, reserves, and carbon dioxide emissions for different fuels and energy sources. The portal also provides access to EIA’s entire library of international reports, articles, and analyses, including Country Analysis Briefs.

Find data in fewer steps.

Users can select and explore prepopulated, fuel-specific default tables that provide annual data on energy production, consumption, trade, and reserves. Data options allow users to change time frequency and energy units, add sources and activities, and select other countries for comparison.

Customize data tables.

Customizable data tables allow users to sort by energy source and activity or by country and region. Users can now see and select countries within continental regions, International Energy Agency regions, and economic groups.

Tailor visualizations to user needs.

Animated maps allow users to see how trends in energy production, consumption, reserves, imports, and exports have changed over time. Other visualization options include heat maps, bubble maps, and time series.

Export data easily.

EIA’s international information is easily downloaded as data, maps, and charts for presentations, reports, and spreadsheets. Data downloads are available in CSV and JSON formats. Application Programming Interface (API) keys are available with EIA’s Microsoft Excel and Google Sheets add-ins, which make regular data updates much easier. Users can also download static versions of charts and tables in PDF or PNG image files.

The post Introducing the U.S. Energy Information Administration (EIA) redesigned International Energy Portal appeared first on Data.gov.

Although electric system balancing authorities covering most of the United States have released real-time information on grid operations since the late 1990s, EIA’s Hourly Electric Grid Monitor expands the availability of data to the entire contiguous 48 states, and makes it available in a consistent format from a single source.

Among other applications, the data can be used to provide timely information on electric system recovery after power interruptions and to help evaluate the effects of renewable energy, smart grid, and demand-response programs on power system operations.  The tool allows you to visualize and analyze:

• Total U.S. and regional electricity demand on an hourly basis
• The varied mix of energy sources used to generate electricity at different times and locations
• The hourly flow of electricity between electric systems
• The wide variety in electric systems’ daily demand shapes and the seasonality of daily demand patterns
• The extent to which electric systems rely on internal and external sources of supply to meet their demand
• Potential stress on electric systems when actual demand significantly exceeds forecasted demand
• Total hourly flows of electricity with Canada and Mexico

Have fun exploring!

The post Hourly Electric Grid Monitor reports new information on U.S. electricity demand, net generation, and interchange collected by the U.S. Energy Information Administration appeared first on Data.gov.

As described by the USCG, “the Marine Casualty and Pollution Data files provide details about marine casualty and pollution incidents investigated by Coast Guard Offices throughout the United States. The database can be used to analyze marine accidents and pollution incidents by a variety of factors including vessel or facility type, injuries, fatalities, pollutant details, location, and date.  The data collection period began in 1982 for marine casualties and 1973 for polluting incidents, and is ongoing. Documentation includes entity and attribute descriptions along with suggested solutions to general marine pollution, vessel casualty, and personnel injury and death questions.”

Visitors to the USCG Homeport data download site should note that there are three files available fore download, but it is the second file on the list (named MISLE_DATA.zip) that contains all available marine casualty data from January 2002 – July 2015.  The files extracted from MISLE_DATA.zip can be opened with most standard spreadsheet editing software programs.

Source: USCG Marine Casualty and Pollution Data, https://homeport.uscg.mil/missions/investigations/marine-casualty-pollution-investigations

Data.Gov Links Related to this topic:

USCG Marine Safety Information Data, https://catalog.data.gov/dataset/marine-safety-information-data

USCG Facility Pollution Database, https://catalog.data.gov/dataset/uscg-facility-pollution

U.S. Coast Guard closed and open incident investigations, 2002 through 2015 (part year).

The post Safety at Sea – U.S. Coast Guard Marine Casualty and Pollution Data for Researchers appeared first on Data.gov.

References:

U.S. Coast Guard NAIS and Data Requests: https://www.navcen.uscg.gov/
E. J. Hilton, B. Kynard, M. T. Balazik, A. Z. Horodysky, and C. B. Dillman. 2016. Review of the biology, fisheries, and conservation status of the Atlantic Sturgeon, (Acipenser oxyrinchus oxyrinchus Mitchill, 1815). Journal of Applied Ichthyology, 32: 30-66.

NOAA Fisheries, Species Directory, Atlantic Sturgeon: https://www.fisheries.noaa.gov/species/atlantic-sturgeon

Data.Gov Links related to this topic:

Green Sturgeon Acoustic Monitoring: https://catalog.data.gov/dataset/green-sturgeon-acoustic-monitoring

White Sturgeon Distribution, Pacific Northwest: https://catalog.data.gov/dataset/white-sturgeon-distribution-pacific-northwest-updated-march-2006

USGS National Hydrography Dataset: https://catalog.data.gov/dataset/usgs-national-hydrography-dataset-nhd-downloadable-data-collection-national-geospatial-data-as

The post Of Sturgeon and Ships appeared first on Data.gov.

Data.gov provides easy access to government datasets covering a wide range of topics—everything from weather, demographics, health, education, housing, and agriculture. The data is used by the public, students, researchers, journalists, and businesses. Usage of Data.gov has grown steadily over the years, reaching approximately 20 million page views annually.

Data.gov’s tenth anniversary coincides with a major change in the program. On January 14, 2019, the OPEN Government Data Act, as part of the Foundations for Evidence Based Policymaking Act, became law. The OPEN Government Data Act makes Data.gov a requirement in statute, rather than a policy. It requires federal agencies to publish their information online as open data, using standardized, machine-readable data formats, with their metadata included in the Data.gov catalog. Data.gov is working with an expanded group of federal agencies to include their datasets in Data.gov as they implement the new law. In addition, the law requires that GSA work with the Office of Management and Budget and the Office of Government Information Services to establish an “online repository of tools, best practices, and schema standards to facilitate the adoption of open data practices across the Federal Government.” This new repository, which will be an update and expansion of Project Open Data, will also be available on Data.gov.

Data.gov marks its tenth anniversary with a strong foundation in a new statutory requirement and expectations for an expanded scope with data from additional federal agencies and for sharing updated tools and resources in a new online repository. Opening more data will continue to ensure that the best sources of information are accessible to the public and support the goals of an open and transparent government.

The post Data.gov at Ten and the OPEN Government Data Act appeared first on Data.gov.

Harvesting for most federal agency dataset sources occurs on a daily, automated basis. As agencies update their metadata inventories, the Data.gov catalog is updated at the next scheduled harvest. As a result, the total number of datasets on the Data.gov catalog normally fluctuates. In the coming weeks and months, we expect to see an increase in datasets provided by a greater number of federal agencies under the recently enacted Open Government Data Act (Title II of the Foundations for Evidence-Based Policymaking Act, Pub. L. 114-435). We will provide updates on our progress in working with the federal agencies in providing access to more federal data under the statute.

The post Duplicate Entries Removed from Data.gov Catalog appeared first on Data.gov.

Links to Data.Gov datasets

USGS Map service: National Shoreline Change – Historic Shorelines by State: https://catalog.data.gov/dataset/usgs-map-service-national-shoreline-change-historic-shorelines-by-state4a6c3

USGS Map service: National Shoreline Change – Offshore Baseline: https://catalog.data.gov/dataset/usgs-map-service-national-shoreline-change-offshore-baseline01dc6

USGS 2010 Topographic Lidar of the Channel Islands, California: https://catalog.data.gov/dataset/2010-u-s-geological-survey-usgs-topographic-lidar-channel-islands-california78b1a

USGS Bathymetry–Offshore of Carpinteria, California: https://catalog.data.gov/dataset/bathymetry-offshore-of-carpinteria-californiaf4193

California 2010 Lidar Coverage, USACE National Coastal Mapping Program: https://catalog.data.gov/dataset/california-2010-lidar-coverage-usace-national-coastal-mapping-program

USGS: A seamless, high-resolution, coastal digital elevation model (DEM) for Southern California. https://catalog.data.gov/dataset/a-seamless-high-resolution-coastal-digital-elevation-model-dem-for-southern-californiaf1e9e

Reference: Johnson, S.Y.,  G.R. Cochrane, N.E. Golden, P. Dartnell, S.R. Hartwell, S.A.Cochran, and J.T. Watt. 2017. The California Seafloor and Coastal Mapping Program – Providing science and geospatial data for California’s State Waters.  Ocean & Coastal Management. 140, pages 88-104.

Detailed Geology and Geomorphology, Hueneme Canyon, California. U.S. Geological Survey. Full size at: https://pubs.usgs.gov/sim/3225/sim3225_sheet11.pdf

The post The California Seafloor and Coastal Mapping Program benefits from shared data appeared first on Data.gov.

The data available for download through the Northeast Ocean Data portal ranges from physical oceanographic characteristics (like shoreline classification and seafloor depth measurements) to administrative boundaries, to animal habitats and species abundance, to data on overlapping ocean uses among diverse user groups such as aquaculture, marine transportation, recreational boating/fishing, and energy production and transmission.  Multiple relevant external data sources such an real-time weather forecasts can also be accessed through the portal.  Users can also view, edit, and download maps on topics such as water quality, commercial fishing, energy and infrastructure, and the presence of animal groups like fish, birds, or marine mammals.  Two example maps are shown below, one displays navigation-related features and the other features marine mammal abundance.

Map of navigation-related features in the northeastern U.S., including channels, shipping lanes, and restricted areas. From the Northeast Ocean Data Portal.

Map of marine mammal distribution areas along the U.S. Northeastern seaboard. Red colors indicate higher recorded abundance. From the Northeast Ocean Data Portal.

For information on a west coast regional data project check out the post on the California Seafloor and Coastal Mapping Program.

If you’re interested only in  vessel-traffic specific information check out the post on Visualizing ocean traffic on Marine Cadastre.

The post Northeast Ocean Data Portal – Providing a Common Picture of New England’s Ocean Uses appeared first on Data.gov.

Open data from the U.S. Government is an important national resource, serving as fuel for innovation and scientific discovery. It is central to a more efficient, transparent, and collaborative democracy. Since President Obama’s first day in office, his commitment to ensuring all Americans reap the benefits of public open data has been unwavering. Throughout the Administration, the U.S. Government has implemented policies to open its data, including:

• Presidential Memorandum on “Transparency and Open Government”
• Open Government Directive
• Executive Order 13642 “Making Open and Machine Readable the New Default for Government Information”
• OMB Memorandum M-13-13: “Open Data Policy—Managing Information as an Asset.”

2015 was a year of great advancement in open data initiatives. Progress on Project Open Data has been growing each year, and is expected to continue as the Administration continues to prioritize open data because of the value returned to society. Federal agencies continue to release and improve their open data for more citizens, non-profits, and companies. You can see how agencies are doing over time, with quarterly evaluations by agency on the Project Open Data Dashboard.

As we look towards the work to be done in 2016, we also celebrate open data milestones and successes over the past year:

• Chief Data Roles. Central to a data-driven government is committed leadership dedicated to data improvements and comprehensive information management. In 2015 the White House welcomed the first-ever U.S. Chief Data Scientist and Deputy Chief Technology Officer for Data Policy, DJ Patil. In 2015, several Federal Agencies, including the Department of Commerce, the Environmental Protection Agency, and General Services Administration, followed suit, adding Chief Data Officers and Chief Data Scientists to their rosters.

• Open Data Making a Difference. Open data was a key ingredient to addressing pressing policy issues in 2015, including: helping students and families better evaluate college choices with a new College Scorecard; making it easier for communities to implement the Fair Housing Act with the Affirmatively Furthering Fair Housing Assessment Tool; and enhancing trust between communities and law enforcement with the White House Police Data Initiative.

• Open Data Updates and Improvements. 2015 also saw many expansions on pivotal open governmental datasets, such as: new U.S. Extractive Industries Transparency Initiative federal production and federal revenue by company data; new Consumer Financial Protection Bureau Consumer Complaint Database tools (API); and various OpenFDA updates, including direct downloads that enable citizens to use data on food, drug, and medical device safety. The Federal Government also provided new platforms for exploring government data, including: Open.nasa.gov, USPTO’s PatentsView, and Landsat on AWS.

President Obama with Chief Technology Officer team members Ryan Panchadsaram and Corinna Zarek, and U.S. Chief Information Officer Tony Scott viewing Analytics.usa.gov

• Website Dashboards. Last year two new dashboards were also created to better evaluate the health of our Federal government websites: Analytics.usa.gov tracks how people are interacting with the government online and Pulse.cio.gov helps the government and public observe progress on HTTPS and Web Analytics.

• Public Engagement around Open Data. Federal agencies featured open data efforts at dozens of hackathons, data jams, and conferences including the 2015 Health Datapalooza,Transportation Datapalooza, Third Annual Safety Datapalooza, Mental Health Hackathons, an Accessibility Hackathon, a White House Mapathon, and eight Open Data Roundtables.

One common denominator of these successes is that open data is a collaborative process, involving Federal government, state and local governments, academia, private sectors, and civil society. It takes a village to transform open data into actionable knowledge and societal value. Like data science, open data is a team sport, united by the shared purpose to responsibly unleash the power of data for the benefit of the American public and maximize the nation’s return on its investment in data. Together in 2016 and beyond, “Team Open Data” will continue transforming open data into knowledge, into action.

(Graphic credit: Radhika Bhatt, U.S. Department of Commerce Data Service)

Open data remains a priority in 2016, offering both government and citizens information to make better decisions and a way for Americans to interact with their government. For upcoming events related to open data, please check out the Data.gov/events calendar.

The post Open Data: Empowering Americans to Make Data-Driven Decisions appeared first on Data.gov.

In May 2009, Data.gov was an experiment. There were questions: would people use the data? would agencies share the data? and would it make a difference? We’ve all come a long way to answering those questions, starting with only 47 datasets and having 105,000 datasets today.  We realized that this was never simply about opening up government data, but rather about growing and nurturing an open data ecosystem to improve the lives of citizens.

Putting the Data to Work

Hundreds of companies, like those featured at the Open Data 500, are using data to create services and products.  From agriculture to housing—businesses of all shapes and sizes are reliant on government data. During the last five years, there have been moments when someone was poised to use open data at a moment of crisis or opportunity. Some of these highlights include:Arc hurricane

• The Red Cross, working with the Department of Transportation, released a Hurricane App just as Hurricane Sandy touched ground—helping 700,000 people be safer that first day alone
• Organizations have come to life around helping others and getting them data to inform their consumer decisions
• A new level of digital democracy was reached when India and the United States, and later Canada and Ghana, created the Open Government Platform, a shared, open source system for open data portals
• City, county, and state data was integrated with federal data to make Data.gov a truly a national open data portal

People today are using open data in even more creative ways. Civic hackers, designers, and developers, young and old, are being educated and connected through Code for America, Code Academy, and others that give them the tools to put the data to work and growing that ecosystem of developers and consumers of the data.

Making More Data Available

Much of the first year of Data.gov was spent on gathering the fuel for the ecosystem—open data. Open data teams at each agency and Data.gov worked together to change the culture from a “how will we release this data” to “what is the impact on Americans when we release this data.” With clear direction from the White House in the Presidential Transparency and Open Government memo, the Digital Government strategy, and the Executive Order for Open Data, people at 175 agencies and 77 other organizations today continue to release more data, more accessible data, and new web services and APIs.2 In using social media, sponsoring data jams and paloozas, creating and moderating the Open Data Stack Exchange, and transparently sharing code and issues on Github, Data.gov and data publishers are listening to and working directly with researchers, innovators, data scientists, and data journalists to create a better platform with more data every day.

Making a Difference

So, has the release of government data helped to improve the lives of Americans?  In just the last three years, Data.gov has grown to include:

• 4.5 million unique visitors
• 105,000 data collections
• 447,000 data resources
• 227 organizations publishing data
• 4700 people attending the International Open Government Data Conference online and in person
• 21 topics
• Citizens from 195 countries have visited the site

Clearly, the answer is that people are working with open data, but there is still much more to do. Groups like the Sunlight Foundation and data journalists help to advocate for the release of new data in machine-readable formats, and citizen groups create activities and initiatives around the data. As one example, behind the topics highlighted on Data.gov (such as climate, health, safety, and ocean), are communities of people from around the country and around the world who strive together to get specific data released to help solve humanity’s challenges. The community behind Agriculture.Data.gov hosted an event last year that led to the formation of the Global Open Data for Agriculture and Nutrition (GODAN) initiative that focuses on food security and mitigating hunger in developed and developing nations.

What Is Your Story?

You’ve heard here about some of the open data stories powered by data on Data.gov.  Now, we are issuing a call for YOUR story. Tell the world what open data means to you.  What have you built? How have you made a difference? What still needs to be accomplished?  Answer the call for Open Data Stories: what we’ve accomplished, understood, and are still exploring after five years. Post a video, write a blog, or send a tweet (#OpenDataStory) and let us know.  We will connect the stories and highlight what you are doing. Be creative, be innovative, and let all of us celebrate your story.

Each person makes a difference—whether you are a researcher, developer, data scientist, business owner, innovator, data publisher, or citizen—each part of the ecosystem is important in our shared vision of making the world a better place through transparency, openness, and knowledge. Be part of the future and celebrate how far we’ve come—share your story, share your vision.

The post Five Years of Open Data—Making a Difference appeared first on Data.gov.

Data.gov is also launching a new data catalog on an open source data management system called CKAN  The new catalog features a number of enhancements, such as an improved search that helps you find all the datasets for a particular location, better sorting and tagging of datasets, and more robust metadata. Data.gov now has one unified data catalog based on an open source standard that will make it easier to federate with other federal agency catalogs, as well as those of states, cities, and counties. 

The new CKAN catalog will also enable the central implementation of the Open Data Policy, as it will easily harvest the data inventories that federal agencies will be creating under the policy. The result will be an easy-to-use, comprehensive catalog that will allow citizens, developers and others to fully take advantage of the vast array of federal data that affects the daily lives of citizens. 

We also released new tools on Project Open Data that will help agencies easily meet the requirement of the policy, while laying the foundation for the new Data.gov infrastructure across government.

Today is just the first step in a number of dramatic improvements in Data.gov scheduled for the months ahead. If you are a developer and want to follow along on the progress, check out the Project Open Data repository on GitHub. Be sure to check out the government-wide APIs and new catalog and stay tuned! 

The post Data.gov Launches New Catalog and APIs appeared first on Data.gov.

The US Department of Veterans Affairs originally launched Blue Button with industry and non-profit collaboration.  It’s a simple, common-sense idea—people should be able to access and download their own health information—with the potential for a big impact.

As President Obama said in August 2010, “For the first time ever, veterans will be able to go to the VA website, click a simple blue button, and download or print your personal health records so you have them and can share with your doctor outside of the VA.”

Why can’t the same common-sense concept be applied to the energy industry with a “Green Button”?  Consumers should have access to their energy usage information.  It should be easily downloadable and in an easy-to-read format offered by their utility or retail energy service provider.

So today at GridWeek, I challenged the smart grid ecosystem to deliver on the vision of Green Button and provide customers access to their energy usage information electronically.  With this information at their fingertips, consumers would be enabled to make more informed decisions about their energy use and, when coupled with opportunities to take action, empowered to actively manage their energy use.

Furthermore, making this information available—in simple standard formats—will help spur innovative new consumer applications and devices from entrepreneurs, big companies, and even students.  Imagine being able to check your air conditioner from your smartphone or having a clothes dryer that saves money for you automatically during critically hot days or simply getting some helpful customized hints on how best to save energy and money in your house or apartment.

This concept of encouraging customer access to electronic energy usage information is part of the Administration’s Policy Framework for a 21st Century Grid. This framework, launched at a White House event in June 2011, highlights the value of empowering consumers with enhanced information to save energy, ensure privacy, and shrink bills.

The vision of consumer access to energy data is shared by the National Association of Regulatory Utility Commissioners (NARUC), which recently passed a resolution endorsing smart grid principles that include the importance of providing consumers with affordable and timely access to their own energy use data.

And if modeled after a successful program like Blue Button, Green Button has the potential to deliver untold benefits to consumers and utilities alike.

Since its release, Blue Button has been adopted by Medicare, the Department of Defense, and private sector health care organizations.  Software developer challenges have spurred applications that expand on the Blue Button’s promise by helping consumers use their data to stay healthy and manage their care.

Through a collaborative effort, we can build an open-reference implementation of a Green Button, based on national standards for the smart grid.  If the health industry can work together through Blue Button to make this world a better place, then the energy industry can do so through Green Button. 

Let’s get to work.

Aneesh Chopra is US Chief Technology Officer

Original blog posted at: http://www.whitehouse.gov/blog/2011/09/15/modeling-green-energy-challenge-after-blue-button

The post Modeling a Green Energy Challenge after a Blue Button appeared first on Data.gov.

Under the third National Action Plan, the United States is committed to more than 40 new or enhanced open government initiatives. Open data is a core component of open government, and Data.gov is excited to be part of the U.S. National Action Plan commitment for the third time.

For the third NAP, Data.gov will focus on continuing efforts to improve the use of public feedback on what federal data needs to be released or made more accessible, as part of its work with agencies to implement the federal Open Data Policy. On a daily basis, Data.gov receives user feedback from Americans who rely on federal open data to advance their interests in commerce, health care, education, safety, and much more. Under the third NAP, Data.gov will work with the Office of Management and Budget to establish user-friendly feedback mechanisms to connect data users directly to agencies, in order to ensure that agencies receive and act promptly on public requests and suggestions. In addition, Data.gov will support NAP commitments on developing national open data guidelines and making municipal data more accessible to citizens as part of the Municipal Data Network. We look forward to continuing our role in expanding access to government data and delivering on an international commitment for open government on behalf of the United States.

The post The Third U.S. Open Government National Action Plan appeared first on Data.gov.

In a continued effort to provide key datasets that support the process of developing resilience to climate change, eight additional datasets are now available in the Water theme.

Those include:

• NOAA Next Generation Radar (NEXRAD) Products

• Severe Weather Data Inventory (SWDI)

• State of the Climate Monthly Overview – National Snow and Ice

• U.S. 15 Minute Precipitation Data

• U.S. Hourly Precipitation Data

• United States Climate Reference Network (USCRN) Processed Data

• GLDAS Noah Land Surface Model L4 3 hourly 0.25 x 0.25 degree Version 2.0 V020

• GLDAS Noah Land Surface Model L4 monthly 0.25 x 0.25 degree Version 2.0 V020

The post Water Theme Adds Eight New Datasets appeared first on Data.gov.

The total number of datasets on the front page of the catalog changes regularly as datasets are added or deleted at the agency metadata source. The total number includes many collections, such as a time series dataset that has periodic datasets across a number of years. A collection that includes thousands of datasets will only count as “1” dataset in the overall total.

You may have noticed significant changes recently in the total number of datasets on the Data.gov catalog front page. This fluctuation is due to a technical problem associated with the harvesting process that is creating duplicate datasets. The Data.gov team is working on carefully deleting the duplicates and restoring full functionality of the harvester. Currently, we are updating the catalog by initiating harvests of individual agency sources on a case by case basis as we work on restoring automatic daily harvesting for all sources.

We are also working on plans to upgrade the software that powers the Data.gov catalog, which is currently based on an older version of CKAN, the open source platform used by many national data catalogs. We will continue to provide updates on the status of the harvester and our migration of the catalog to the current version of CKAN. If you have any questions, please feel free to contact us.

The post Data.gov Catalog Update appeared first on Data.gov.

In less than a week since Popular Choice voting began in our Apps for the Environment challenge, you already placed more than 1200 votes in support of your favorite green apps. We’re excited by the response from the community, but not surprised after people from around the country already submitted almost 100 ideas for apps and created 38 final products that both use EPA data and promote public health. By listening to and working with the community from the beginning, together we’ve gained the momentum in helping developers and entrepreneurs use our environmental data to create innovative services that can make an impact.

We didn’t get here overnight, however, the contributions and leadership of people from coast to coast brought the Apps for the Environment mission into our local communities. The CleanWeb Hackathon recently held in San Francisco brought together a diverse array of engineers, data providers, start-ups, designers, and investors to see what kind of useful apps could be created around energy and environment data over the course of a weekend. One-hundred and seventy-five people signed up for 100 spots, including a specialist from our own EPA Region 9 office, to innovate across 60 different data sets.

Despite it being a separate event from Apps for the Environment, multiple developers who participated also submitted to our challenge. This was a big step toward building a sustainable relationship between EPA, the Department of Energy, and developers.

Thank you all for your continued participation, and remember, you can still vote for Popular Choice App for the Environment until Oct. 7 – make your voice heard, join our community as a supporter, and let us know how we can work together better as we bring innovation from your ideas to the palms of hands.

Robin L. Gonzalez, Acting Director
Office of Information Analysis and Access Office of Environmental Information, EPA

The post More than 1000 votes for Popular Choice GreenApp Fueled by Communities, Engagement appeared first on Data.gov.

Many of the Task Force’s recommendations emphasize the opportunity for departments to better use data and technology to build community trust. As a response, the White House has launched the Police Data Initiative, which has mobilized 21 leading jurisdictions across the country to take fast action on concrete deliverables responding to these Task Force recommendations in the area of data and technology. Camden is one such jurisdiction.

By finding innovative work already underway in these diverse communities and bringing their leaders together with top technologists, researchers, data scientists and design experts, the Police Data Initiative is helping accelerate progress around data transparency and analysis, toward the goal of increased trust and impact. Through the Initiative, key stakeholders are establishing a community of practice that will allow for knowledge sharing, community-sourced problem solving, and the establishment of documented best practices that can serve as examples for police departments nationwide.

Like many police departments, Camden wants to use smarter, more data-driven ways of improving community policing efforts and reducing uses of force. However, challenges with technology systems often get in the way. For instance, the Camden County PD cobbles together 41 systems that have individual value but are not designed to work together, requiring their beat officers to enter the same data multiple times. In this environment, meaningful data analysis ends up being extremely difficult to conduct, requiring analysts to spend more time on extracting the data than on critical analysis.

The President’s Police Data Initiative has assembled a volunteer team of technology experts who are also in Camden today. The team will spend two days in Camden on a design sprint, engaging directly with front line officers, detectives, crime analysts and department leadership to help envision what a truly effective technology system could look like. The two-day deployment will help the team consider best practices and address specific technology questions as they arise, enabling departments like Camden to find the solutions that most fit their needs.

By upgrading its technology practices, the Camden County PD will have more efficient data supply chains, and will be better positioned to use that data to improve its internal operations and to identify and solve policing problems in a more timely manner. The lessons learned in Camden can help law enforcement around the country both by example and also directly since some of the development work can be shared though open source best practice.

Camden is just one of 21 communities currently participating in our Police Data Initiative. Through this effort, local police departments and other participants are responding first to Task Force recommendations within two streams of work:

• Using open data to increase transparency, build community trust, and support innovation
• Better using technology, such as early warning systems, to identify problems, increase internal accountability, and decrease inappropriate uses of force

The effort has focused on specific actions law enforcement agencies can take to make progress in these two areas. The collaboration has generated multiple commitments to action and the White House is working with agencies and key enabling partners now to drive quick implementation.

Police departments participating in the White House Police Data Initiative:

Graphic created by Presidential Innovation Fellow Christopher Wong

Commitment highlights include:

1. Use open data to build transparency and increase community trust.

• All 21 police departments have committed to release a combined total of 101 data sets that have not been released to the public. The types of data include uses of force, police pedestrian and vehicle stops, officer involved shootings and more, helping the communities gain visibility into key information on police/citizen encounters.
  • Code for America and others are helping on this. For information on how Police Departments can jumpstart their open police data efforts, click here.
• To make police open data easy to find and use, the Police Foundation and ESRI are building a public safety open data portal to serve, in part, as a central clearinghouse option for police open data, making it easily accessible to law enforcement agencies, community groups and researchers.
• Code for America and CI Technologies will work together to build an open source software tool to make it easier for the 500+ U.S. law enforcement agencies using IA Pro police integrity software to extract and open up data.
• To make it easier for agencies to share data with the public about policing, Socrata will provide technical assistance to cities and agencies who are working toward increased transparency.
• To help this newly released data come alive for communities through mapping, visualizations and other tools, city leaders, non-profit organizations, and private sector partners will host open data hackathons in cities around the country. In New Orleans, Operation Spark, a non-profit organization that teaches at-risk New Orleans youth software development skills, will work with data opened by the New Orleans Police Department at a weeklong code academy.
• Presidential Innovation Fellows working with the U.S. Chief Technology Officer and Chief Data Scientist will work collaboratively with key stakeholders, such as Code for America and the Sunlight Foundation, to develop and release an Open Data Playbook for police departments that they can use as a reference for open data best practices and case studies.
• The Charlotte-Mecklenburg Police Department is working with the Southern Coalition for Social Justice to use open data to provide a full picture of key policing activities, including stops, searches and use-of-force trends, information and demographics on neighborhoods patrolled, and more. This partnership will build on a website and tools already developed by the Southern Coalition for Justice which provide visualization and search functions to make this data easily accessible and understandable.
• The International Association of Chiefs of Police, the Police Foundation, and Code for America have committed to helping grow a community of practice for law enforcement agencies and technologists around open data and transparency in police community interactions.

2. Internal accountability and effective data analysis.

• While many police departments have systems in place, often called “early warning systems,” to identify officers who may be having challenges in their interactions with the public and link them with training and other assistance, there has been little to no research to determine which indicators are most closely linked to bad outcomes. To tackle this issue, twelve police departments committed to sharing data on police/citizen encounters with data scientists for in-depth data analysis, strengthening the ability of police to intervene early and effectively: Austin, TX; Camden, NJ; Charlotte, NC; Dallas, TX; Indianapolis, IN; Knoxville, TN; LA City; LA County; Louisville, KY; New Orleans, LA; Philadelphia, PA; and Richmond, CA.
• The University of Chicago will provide a team of five data science fellows from the Eric and Wendy Schmidt Data Science for Social Good program to work with 3-4 police departments over a 14 week engagement, starting in late May, to begin to prototype data analysis tools that will help police departments identify the behaviors most indicative of later problems.
• The Oakland Police Department, which has deployed body worn cameras for over four years, has partnered with a team of researchers at Stanford University to build automated tools to comb through the audio to surface police/citizen encounters that either went particularly poorly or went particularly well. This will allow the Oakland PD to quickly identify problems and also to lift up real world examples of the great police work that happens every day. The Stanford team is also researching ways that body worn camera data can be used to track and inform the effectiveness of training in the field, using the camera data to see whether the classroom experience translated effectively to encounters on the street.
• The Department of Justice and the Police Data Initiative stakeholders will work with universities and other research partners to identify opportunities to coordinate body worn camera research to help avoid unintended overlap, maximize the coverage of research topics and increase cross-learning. Additionally, the Police Data Initiative will work with cutting-edge leaders in advanced video analysis to identify opportunities to help police departments maximize the value of the thousands of hours of video body worn cameras will produce.

We are excited about the Police Data Initiative as a fast, tangible, collaborative response to recommendations of the Task Force on 21st Century Policing and we are glad that so many cities have stepped forward to work towards better use of data and technology in ways that will build community trust and impact.

Megan Smith is the United States Chief Technology Officer (CTO) in the Office of Science and Technology Policy. Roy L. Austin, Jr. is Deputy Assistant to the President for Urban Affairs, Justice and Opportunity in the Domestic Policy Council.

The post Launching the Police Data Initiative appeared first on Data.gov.

Last week, the Obama Administration focused on a specific climate risk – the risk to the food supply – and the ways data could be used to help increase “food resilience.” The goal is to make data, computational power, and analytic tools available to help food producers, distributors, and inspectors keep the food supply reliable and robust.

As part of that effort, last Friday the GovLab and the U.S. Department of Agriculture (USDA) co-hosted an Open Data Roundtable on food resilience to bring together government officials, companies, and nonprofits to improve the use of data on climate and agriculture. Like the Roundtable we hosted with the White House and the Department of Commerce in June, this event was designed to promote a dialogue between government agencies that supply data and the companies and organizations that use it. The ultimate goal of all our Roundtables is to make open government data more relevant, accessible, and actionable.

While it may be a few weeks before all the suggestions from last Friday’s Open Data Roundtable at USDA are compiled into a report, companies and USDA officials came away with a better understanding of how public data is gathered, disseminated and processed by users.  The meeting was part of a series of discussions initiated by USDA.  Undersecretary and Chief Scientist Cathie Woteki, Deputy Undersecretary Ann Bartuska and Chief Information Officer (CIO) Cheryl Cook listened to and participated in discussions aimed at making it easier for crop, weather, soil and other information gathered by USDA to be provided to those who need it, here in America and abroad.

Some of the recommendations made by the group during two breakout sessions:

• Things change, so develop an ongoing dialogue between users and providers of the data. Reach out to users (customers) to determine what they value and what they use.
• Standardize data across sets so that, for example, the definition of something as simple as “corn” is universal.
• Develop a better system to enable a searcher to get what he or she wants without having to sort through reams of data first.
• Have a “crosscut” system to keep users from having to go to multiple agencies or places for portions of the data they need.  Provide “one stop” service.
• Provide better service to those who request information under the Freedom of Information Act.
• Standardize tables to make it easier for business to convert raw data into information it needs.
• Create a “data concierge” to assist users, improve the search function of @data.gov.
• Set long terms improvement goals and develop a roadmap to get there.  Improve geospatial data and make it the “highest quality possible.”
• Build an app that provides a “real time” data alert system, which could also be employed by USDA and other government departments to share information with the public during times of crisis or disaster.
• Leverage data that is already available, such as drought monitoring and let it be used for expanded purposes, such as predictive analysis to drive efficiencies and insight to create a smarter “AgriBusiness”.

The White House recognized the value of the Open Data Roundtables in the U.S. Open Data Action Plan, released on May 9, 2014, which described the Roundtables as helping to “support innovators and improve open data based on feedback.” The Plan notes that “Specific, actionable feedback from these sessions [the Roundtables] and others has the potential to improve descriptions, formats, and accessibility of government data.”

At the conclusion of the meeting, Dr. Woteki pledged to continue her effort to provide more, free access to USDA’s scholarship and data.  She noted that these efforts build on an Open Data event spearheaded by the G-8 countries at the World Bank over a year ago, and continued this week as African leaders from many nations came to Washington to discuss agricultural and other issues.  “Agriculture has a growing role in open data implementation, because we face a growing challenge to produce more food for more people, and to provide the services they expect,” said Dr. Woteki. “The only way to achieve this is through collaborative work, innovation sharing, and research.”  USDA CIO Cook talked about how data was crucial to her previous jobs with the State of Pennsylvania, then as a Federal official in the Clinton and Obama administrations.  “As someone who has been a data producer and a data user,” said Cook, “I understand the need to consolidate data so it can be used for multiple purposes to help our clients achieve their goals. We support public-private partnerships, open data and the need to know who is using our data and how, so we can serve them better.”

Originally posted at USDA.gov.

The post Participants in Open Data Roundtables Guide USDA on Ways to Provide High Quality Data to Users appeared first on Data.gov.

(Mar 21, 2015) Today I was honored to represent NASA at the Climate Data Initiative launch at the White House. Along with NOAA, Google, and many partners from other government agencies and the private sector, we opened the climate.data.gov gateway with more than 100 curated, high-quality datasets, web services, and tools that can be used by local officials, emergency management personnel and innovators to help communities prepare for the effects of our changing climate change.

For decades, NASA has shown great leadership in assembling a one-stop-shop of space-based Earth science and climate change data. across all disciplines of Earth Systems Science. Given our experience with this type of large-scale data integration, we welcomed the challenge to take on a leadership role in the first phase of the Climate Data Initiative, which integrates government-wide federal datasets across all disciplines that may have an impact on coastal flooding, and to make these datasets easily available.

At NASA, we are working proactively to protect our own national assets along the coasts from the potential impacts of climate change. From Wallops Island, Virginia, to the Space Coast of Florida, we are bringing together our Earth science researchers and our strategic infrastructure managers to provide climate-related decision support in the face of sea level rise and other serious coastal changes. The launch of climate.data.gov opens the door for citizen scientists to get involved in this type of work, in support of coastal communities around the nation.

NASA has a long history of engaging generations of global citizens to use publicly available data to design creative solutions to improve life on Earth, and enable humans to live and work in space. At the Climate Data Initiative launch event, I was delighted to announce that today, NASA and NOAA are together launching a new opportunity for citizens to work with us on this very important topic of coastal flooding.

This coastal flooding challenge is part of NASA’s third International Space Apps Challenge – a two-day global mass collaboration event on April 12-13, 2014. During these two days, citizens around the world are invited to engage directly with NASA to develop awe-inspiring software, hardware, and data visualizations. Last year’s event involved more than 9,000 global participants in 83 locations. This year will introduce more than 60 robust challenges clustered in five themes: asteroids, Earth watch, human spaceflight, robotics, and space technology.

The Coastal Inundation In Your Community challenge is one of four climate-related challenges using data provided by NASA, NOAA and EPA. It invites citizens, students, and all others to create data visualizations and simulations to help people understand their exposure to coastal-inundation hazards. Solutions developed through this challenge could have many potential impacts, including helping coastal businesses determine if they are currently at risk from coastal inundation, and if they will be impacted in the future due to sea level rise and coastal erosion.

I encourage you to roll up your sleeves and get involved in this challenging and energizing event by visiting spaceappschallenge.org.

The Coastal Inundation in Your Community challenge and today’s rollout of coastal flooding data sets are just the beginning. I look forward to the rollout of additional data sets in climate.data.gov, and the opportunity to continue working with this esteemed community of my colleagues at every level of government, our public and private-sector partners and the world’s citizen scientists. And I’m even more also excited about NASA’s upcoming year of major Earth science milestones, including the launch of five missions to study the most important planet in our solar system — ours.

At NASA, we are known for exploring new worlds with rovers on Mars, scientific experiments on the International Space Station, and a vast array of missions exploring our Sun, our neighboring planets and the distant universe. Of all the planets we have explored, none yet have matched the dynamic complexity of our own Earth.

We have a long history of exploring our home planet. For decades we have designed, built, and launched Earth-observing missions to return valuable information about this complex planet we call home. In 2014, we are building on this legacy with five new Earth-observing missions – more than NASA has conducted in a single year in over a decade. These missions will provide global measurements of precipitation, soil moisture, atmospheric carbon dioxide, winds, clouds and aerosols. Your planet is changing. We’re on it.

The author is NASA’s Chief Scientist.

The post NASA Invites Citizens to Collaborate on Coastal Flooding Challenge appeared first on Data.gov.

While the term “open data” isn’t even 20 years old, the author puts the concept in a historical context; the idea that scientific research should be free to all was popularized by Robert King Merton in the early 1940s. Research (which produces data) should be shared freely for the common good.

Fast forward to the early 21^st century. Scientific culture intersects with the burgeoning Internet/Information Technology society. By this time, scientists and even the general public take for granted that scientific research should be available for the public good; in fact, debates arise around length of patent protection for certain products due in part to the assumption that research data is a public resource.

In December 2007, in Sebastopol, California, a group of thought leaders gathered to discuss and define the concept of open public data. By this time, the open-source software movement was in full swing, with people collaborating, or “crowdsourcing,” software by using the Internet as their workspace. The product/software and its ongoing improvements were available for free and visible to everyone in real time via the Internet. Those who met in Sebastapol understood the Internet’s potential and the value of making data, particularly government data, understood and available as a public resource, just as our natural resources are shared for the common good. They actively promoted this idea.

Less than two years later, on his first day in office, President Obama signed the Memorandum on Transparency and Open Government, stating, Information maintained by the Federal Government is a national asset. My Administration will take appropriate action, consistent with law and policy, to disclose information rapidly in forms that the public can readily find and use.

That May, Data.gov was born. As we look forward to our fourth birthday next month, we have learned a great deal – and we’re still on the learning curve. In a few weeks, we will unveil a new catalog and look forward to your feedback on it.

In the meantime, check out “A Brief History of Open Data” in the Paris Tech Review.

Sally Ruth Bourrie of Phase One Consulting Group supports Outreach and Communications at Data.gov.

The post Open Data: A History appeared first on Data.gov.

The post 10 Big Data Sites to Watch – By Gil Press | Foreign Policy appeared first on Data.gov.

Government agencies are investing in strategies to increase access to information created, collected, and held by government. While these strategies typically involve placing information on the web, they sometimes exhibit a misunderstanding of the importance of providing that information in “machine readable” formats. The degree to which information is machine readable, however, is critical to meeting priorities such as open government and open data, and directly influences, and in many cases limits, the uses citizens and other interested parties can make of that information.

Historically, efforts to make government information available to the public have focused on pushing static information about government programs and services to the web. The intended user has been a human who can read, print, and take actions based on reading the material or by engaging in a form-based transaction. In some cases, users were able to query the data or map the results using sophisticated geospatial displays. Access to the data itself, on the other hand, was rarely provided.

This historical practice has been dictated in large part by the capabilities of familiar web technologies (such as HTML and PDF) and a narrow vision of the expected users and uses of the information. Machine readable formats expand that field of vision to new users and new uses and require technologies (such as RDF, XML, and JSON) that may be less familiar in this context, but are still proven and widely used in other arenas (such as web services and content management). Taking the next step to machine readable formats requires government agencies to fully appreciate the potential uses of government information as well as the range of possible formats and the implications of those formats on intended use.

So where to begin? An important starting point is to understand that “machine readable” is not synonymous with “digitally accessible” information.  Scanning a report, the text, graphics, or even rows and columns of numbers, makes it digitally accessible, but a computer still is not really able to “understand” the information. This distinction can be seen in the difference between a magazine cover and a barcode on that cover. A computer cannot directly understand what the picture on the magazine represents, even if it is presented in an online format, but it can read and understand the bar code, using it for identifying the price and tracking the purchase, for example.

Let’s consider the significance of that bar code – a machine readable format that’s universally embedded in products and documents today, but was virtually unseen just 40 years ago. What are some factors that account for its prevalence now?

• Uniformity – The standard format of the UPC (Universal Product Code) established in the 1970’s made widespread adoption possible as tools for printing and processing the UPC could develop around an accepted model.
• Simplicity – The bar code itself is a rather unobtrusive symbol that today has relatively little impact on its attached product. Consumers barely notice it and it’s created without much additional effort.
• Ubiquity – The bar code, which was introduced for grocery store items, now appears on virtually every item that can be purchased or distributed.
• Economy – The cost savings realized through faster, more accurate transaction processing made the bar code a worthwhile investment. The savings soon extended to better inventory management, customer relationships, and theft control (you can’t switch bar codes as easily as you can switch price tags).
• Extensibility – The types and amount of data that could be stored and processed via bar codes expanded its application to areas such as health care (for tracking patient and medication information in hospital settings), distribution (for tracking and expediting shipping and mailing), and scientific research (for banding birds and other wildlife) to name just a few.

Similar factors are now coming into play with the advent of machine readable formats for government data. The expectation of similar benefits in usability, payback, and extensibility should provide the motivation. To realize these benefits, similar conditions also apply. Uniformity and standardization in data formats and processing are needed. Simplicity in creating and embedding the formats must be achieved. Cost advantages must be realized through their use to justify their creation. When these conditions are met, machine readable data become more prevalent leading in turn to increased capabilities.

For these reasons and as commitments to open government and transparency increase, efforts to make information available must include machine readable versions of datasets and not only reports about this information in document form (such as PDF, HTML, and JPG).  Consider a bar chart in a government report. You can read the report in PDF format and understand the analysis the chart provides. However, neither the chart itself nor its underlying data is available in a way that allows further processing of that information. Next generation efforts in opening government must ensure that users have access, for example, not just to a static bar chart image, but also to information about the source of that bar chart and the underlying data itself, much as the magazine bar code described above reveals far more data about the magazine that can be leveraged for additional benefits.

In a practical sense, machine readable information helps government agencies to bridge the gap between “documents” (which are typically static and frozen in their format) and “data” (which may be dynamic and can be open to further processing). By adopting a machine readable perspective, these same agencies can meet their open government and open data objectives more completely, reliably, and responsibly.

Further implications of “documents” versus “data” and the various levels of machine readability, along with the conditions required for broader and deeper implementation, are far beyond the scope of this paper which intends to provide a basic awareness of the main components of machine readable information. To aid in that awareness, the following sections briefly outline the characteristics and differences between documents and data and take a quick look at the special case of syndication formats. It concludes with a glossary of relevant terms. Subsequent papers will address the issues surrounding machine readability and related policies in greater detail.

Machine Readability for Text Documents

PDF (Portable Document Format) is the primary format used to make government information available to the public in document form. While PDF has made it possible for many documents to be available on the web, simply providing a PDF report does not fully address the need to have information about the document itself or its underlying data. For example, the PDF does not necessarily indicate who authored it, where it came from, or the nature of its content. This type of information about the document is often referred to as the “metadata” for the document. Much as a bar code contains additional data about a product, this metadata provides additional data that can assist citizens and researchers searching by a particular author or on a particular topic. Without access to metadata on the thousands of government documents on the web, citizens are essentially looking for a needle in a haystack. There are a number of tools available to extract that kind of information from many kinds of PDF documents and to make it available for use by application developers to ensure that data is available for searching by citizens and others. Thus, PDF documents are a step towards machine readability, but are most useful when enhanced with appropriate metadata.

XML (eXtensible Markup Language) was developed for several purposes, one of which is to make the metadata of documents more directly available.  XML, when properly used, tags information in a document so that computers can automatically extract it to help provide users with capabilities such as searching, browsing, and information discovery.  Along these lines, an important emerging property of XML involves providing mechanisms for tracking versions of a document.  When used, it allows the history of a document to be made available to others — an important capability for tracking legislation, policy guidance, and decision memos.

Producing quality metadata is pivotal to the ability of search tools to find a particular document in response to particular queries. XML does not produce metadata, but is a tool to make it available for use. Producing metadata and then ensuring its use through languages such as XML falls to the document owners and producers and the technologists who support them.

Machine Readability for Data

Machine readability directly influences data usability. Datasets, in particular very large datasets, on their own convey little information to a human. Only when that data is processed in some way — visualized, analyzed, or summarized — does it becomes informative or useful. Thus, to fully realize the potential of open government data, government agencies must release their data in a format that allows processing.  Providing innovators, journalists, and other end users with data in this way makes it possible for them to better understand the raw data, to examine it in ways that meet their interests and responds to their questions. It allows them to drive their businesses; in some cases it becomes their businesses.

The most common machine readable format for data is “Comma Separated Variables” (CSV), which is provided directly by many standard database and spreadsheet products.  CSV stores tabular data in a text-based format, making it easily exchanged by machines.  CSV, like XML for documents, does not inherently make metadata available. There are a number of metadata formats being developed for use along with CSV, but currently these tend to be defined by those developing open data websites, rather than the data providers themselves.

A general problem with CSV is that because it is primarily text-based, it makes it difficult for computers to find common elements between datasets.  If one dataset has a value of “Alaska” and another has one of “AK” then separate information must be retained somewhere that can link these things to each other.  The term “linked data” and the emerging Resource Description Framework (RDF) standard are designed to both enhance the capability of datasets to have their metadata directly attached and also to allow common terms to be related throughout datasets. These features of RDF are making it increasingly attractive for modern open data sites.

Syndication Formats

When information is published in machine readable formats, that information, and especially its metadata, can be shared via “syndication” that enables automatic, continuous feeds of information.  A user, using any one of a number of applications, can request that the information be “pushed” to their machine.  Common syndication formats, such as RSS, Atom and JSON are thus used for sharing machine readable documents, but are not themselves machine readability solutions. They can be viewed more accurately as beneficial outcomes of machine readability, an outcome that most people are familiar with today.

A Glossary for Machine Readability

The following glossary provides basic definitions for terms and acronyms that you may encounter when investigating machine readability. They are arranged alphabetically, not by topic or theme.

API: (Application Programming Interface) When information is made available in any machine readable format, it becomes possible to make that information directly available to programs that request that information over the web.  An API is the way this information is made directly available to other machines.

Atom: Atom is a machine syndication format based on XML.  It is widely used in applications that publish documents or news items, and then syndicate the basic metadata about that document (title, authors, etc.).

CSV: (Comma separated values) A format that stores tabular data (numbers and text) in plain-text form. CSV is an open format and is widely used to store and organize tabular data. Most open government data sites use CSV as the data release format.

Digital Formats: Any format that can be stored, viewed and shared by a computer. While all machine readable formats are also digital formats, not all digital formats are machine readable.

Five-star (linked open) Data: Tim Berners-Lee, inventor of the World Wide Web, proposed a rating scheme for open government data.  He suggested that making data available online was one star, making it machine readable was two-stars, using an open-format was three stars, using RDF format was four stars, and using RDF with links to other datasets was five stars.

HTML:  (HyperText Markup Language) The main markup language for displaying web pages and other information in a web browser. HTML is an open standard by the W3C. The tags and markup of HTML mainly relate to how information is displayed to a human user, not to the information itself. In general, putting a document in HTML, without some form of tagging or metadata, is not considered making it machine readable in a useful sense.

JSON: (JavaScript Object Notation) A machine readable data format derived from the JavaScript language used on many web sites.  It is used for representing simple data structures and associative arrays. As a data serialization format it is language and machine independent. JSON is an open format and is defined in RFC 4627. JSON is also used by Facebook as a way of publishing public social-networking information, which is increasing its use in datasharing and syndication applications.

Metadata: Machine readable data is most useful when information about who produced it, when, etc. is associated with the data. Both documents and structured datasets require metadata to be searchable, sharable, etc.

Open Format: The U.S. Government through the Open Government Directive (http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-06.pdf) defines an open format as “one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.” All of the formats discussed in this document are Open Formats.

PDF: (Portable Document Format) A file format used mainly to represent documents such that layout will stay the same independent of the system environment. As of 2008, Adobe released the PDF format as an open standard. PDF is mainly a human readable format, concerned with layout and organization. Tools for connecting metadata to PDF or extracting the text from it make PDF more machine readable than many other document formats.

RDF: (Resource Description Framework) A data language used to represent data and information as web resources so that the can be “linked” together. RDF is an open standard from the W3C. RDF is increasing in popularity as a data release language as it allows common terms to be linked between datasets.

RSS: (Really Simple Syndication) An XML (or RDF) based language used to publish frequently updated works. Includes both content and metadata about the content (i.e., author and date). An RSS feed is a common syndication method for sharing documents published on the web.

Schema.org: Schema.org is a way of embedding metadata about common objects into other web documents. It is supported by Bing, Google, Yahoo! and other web search engines.  Use of Schema.org enhances the ability of search engines to find machine readable documents and to optimize their search rank.  Use of schema.org is an increasingly important enhancement to machine readable documents due to its role in increasing Search Engine Optimization for online resources.

Syndication Formats: Digital formats used to help publish continuous feeds of information.

XML: (eXtensible Markup Language) A markup language that defines a set of rules for encoding documents in a format that can be both human and machine readable. XML is an open format created and maintained by the W3C. XML is enhanced by proper use of XML schemas or by use of metadata embedded in the XML.

The post A Primer on Machine Readability for Online Documents and Data appeared first on Data.gov.

Project Open Data is an online, public repository intended to foster collaboration and promote the continual improvement of the Open Data Policy. We wanted to foster a culture change in government where we embrace collaboration and where anyone can help us make open data work better. The project is published on GitHub, an open source platform that allows communities of developers to collaboratively share and enhance code.  The resources and plug-and-play tools in Project Open Data can help accelerate the adoption of open data practices.  For example, one tool instantly converts spreadsheets and databases into APIs for easier consumption by developers.  The idea is that anyone, from Federal agencies to state and local governments to private citizens, can freely use and adapt these open source tools—and that’s exactly what’s happening.

Within the first 24 hours after Project Open Data was published, more than two dozen contributions (or “pull requests” in GitHub speak) were submitted by the public. The submissions included everything from fixing broken links, to providing policy suggestions, to contributing new code and tools. One pull request even included new code that translates geographic data from locked formats into open data that is freely available for use by anyone.

These steps may seem small, but they represent a big shift. Behind these actions is recognition of the simple fact that, as a community, we can do more together than we can alone. Project Open Data leverages the ingenuity of innovators everywhere as partners to help the country realize the full benefit of open data.

We are excited to test this new model of policy refinement and implementation. We invite you to add your own contributions to Project Open Data, and hope you will participate in helping to transform our government for the 21^st century.

Here’s how Project Open Data works:

• Contributing:  Project Open Data is a collaborative, open source project.  Both Federal employees and members of the public are strongly encouraged to improve the project by contributing.  Fortunately, contributing is very easy. Simply click the “Improve this content” button at the top of every page, make your edit, and hit “submit.” Your changes will appear once they are approved. Additional instructions can be found here.
• Owners: Project Open Data is managed by the White House Office of Management and Budget (OMB) and Office of Science and Technology Policy (OSTP). As the Federal CIO and CTO, we both plan to be actively involved in this exciting project and are looking forward to merging many more “pull requests” along the way. Members of our teams will also be involved in maintaining the project.
• Approving changes:  In GitHub speak, Project Open Data is actually a collection of different little-p “projects” housed in individual repositories, or “repos.” Each individual project repo will be managed as an open source project – i.e., users can make pull requests (suggest changes).  A repo manager will adjudicate the pull requests (accept, modify, or reject) in a public log on a standard release cycle. Changes to relevant policy areas will be reviewed and approved by relevant policy officials.  We will document and carry out these conversations within Project Open Data through issues and comments.
• Timing: White House owners will have regular “stand ups” to review the pull requests and ensure suggestions are addressed in a reasonable time frame.  Changes that are relatively minor (e.g., typos, minor edits) should be approved fairly quickly, while more substantial changes may take longer to review and approve with necessary White House stakeholders.

Todd Park is the US Chief Technology Officer

Steven VanRoekel is the US Chief Information Officer.

The post Introducing Project Open Data appeared first on Data.gov.

This will be my final blog entry as Program Executive for Data.gov, as I am departing to become the Chief Information Officer for the State of Hawaii. There, Governor Abercrombie and I will work together to bring Hawaii’s data forward and look at ways to continue to collaborate with this team and with all of you. In Hawaiian, “Aloha” means many things, including “Hello,” “Goodbye,” and “Good luck,” so as I depart for the Islands, I wish all of you a fond farewell and the best of luck in the future.

Top Performer

The last few months have been busy and exciting ones for Data.gov.  We continue to make great strides in technical capability and outreach efforts to spread the message of open government data. Data.gov has achieved an overall rating of 9.9 out of 10 on the IT Dashboard, one of the highest in federal government.  We have also received numerous awards, such as being named a finalist in the 2011 ACT-IAC Excellence.gov awards, and being named to CIO Magazine’s annual CIO 100 Awards, as one of the 100 innovative organizations that uses IT effectively to create business value, one of only 4 awarded to the Federal government.

Data.gov Next Generation is Successfully Underway, Data.gov Classic Continues

In May 2011, Data.gov launched Data.gov Next Generation, which transforms the site from simply an online catalog to a cloud-based data services platform that provides benefits in the areas of citizen experience, developer enablement, and agency empowerment.  Rather than going to another site to download the data, you can get to the data right away to analyze, sort, group, and visualize the data live, via the cloud.

Click here for an example. Transforming data into information puts the data to work, allowing citizens to be more informed, make better decisions, and derive greater value from their government.  The datasets on the new platform also feature an open application programming interface (API), making it easier for developers to mash up data and create apps. For fans of the old Data.gov, fear not! The “Classic” version of the catalog will remain available and continue to be updated as new datasets are added.

Communities Are the Future

We continue to add to Data.gov Communities, which provide citizens, government, researchers, and industry with a forum to share ideas and feedback about data on topics that are important to the American public and that tie into the missions of many agencies.  Communities provide discussion forums and opportunities for collaboration and innovation among data consumers, and help to make data more accessible and more relevant by giving users a view of the catalog tailored to their particular interests. 

On June 30, 2011, we launched our latest community, Energy.Data.gov.  The Energy Community is aimed at increasing awareness and improving insight into our Nation’s energy performance, bringing together datasets, tools, and applications using data about energy and how it is used.  Data about alternative fuels, energy efficient buildings, and how to conserve energy at home are available to empower Americans to understand energy issues, including the energy use of the federal government. 

These new innovations are a challenge as we have experienced dramatic cuts to the eGov fund and the Data.gov budgets.  However, in a new paradigm, these communities and some additional services are moving forward because of reimbursements from agencies that support the initiatives and goals of these new communities.  Stay tuned as the Data.gov team, working with other agencies, brings up new communities that focus on the interests of the American public.

Geospatial Integration is Coming Soon

This summer, we will merge Geospatial One Stop (geodata.gov) into Data.gov to consolidate the Federal government’s two primary data catalogs into one site. Bringing the two sites into one will simplify citizens’ search for data, whether geospatial or tabular, while still providing the same visualization (e.g. mapping) capabilities that currently exist at Geospatial One Stop. The consolidation will also save money by eliminating the need for the technical infrastructure of an independent Geospatial One Stop, and by providing citizens and agencies alike with a single source for all federal civilian geospatial data.

The Future is Bright

In the next several months, Data.gov will continue to transform and grow.  Additional Communities are in development, and Data.gov Next Generation will host more and more of the federal agencies’ datasets, so that the government’s data is available in a format that is more useful to citizens, researchers and developers.  As Data.gov continues to improve, it will maintain its position as a world leader in open government data.  Be part of this journey—explore the data, share what you learn, and help to deliver the power of understanding through visualizations, apps, and analyses.

Thanks for the Memories

I would like to thank everyone who attended my Hawaiian-themed farewell party for their kind words and best wishes. The event was a wonderful send-off, and I am grateful for the appearance by the Federal CIO, Vivek Kundra, as well as other Department and Agency CIOs who attended.

In my time as Data.gov Program Executive, I have had an incredible team of over-achievers supporting me – they are the real heroes and heroines of Data.gov and the Open Gov movement. Marion Royal, Hyon Kim, Alan Vandermallie, and Sally Kiel from GSA, Joey Hutcherson from Commerce, George Thomas from HHS, and Jeanne Holm from NASA have all been key to the success of Data.gov and it has been my pleasure to have worked with each of them. Dr. Jim Hendler from the Rensselaer Polytechnic Institute, and Nick Harrigan, Hiko Naito, and the rest of our contractor team have been superb and provided world-class support. It’s been great to be part of the Data.gov team and the global Open Government phenomenon.

Aloha and Mahalo!
Sonny

The post Data.gov’s Journey Continues appeared first on Data.gov.

USDA-NRCS rangeland scientist Emilio Carrillo uses an open data mobile app called LandPKS for sustainable land use management.

Every day, people around the world use data to make decisions.  When heading out of town, most of us use weather apps to check the forecast anywhere in the world before packing our bags.  However, when we travel to far-flung places, we may find ourselves packing food from home because we don’t know what may be available when we arrive.  We have a global, comprehensive, open data set that enables weather forecasting, but not something similar for food and agriculture?

In his first public remarks as head of USDA, Secretary Sonny Perdue noted that “…we want to make decisions based on facts and evidence,” “we want to be data-driven,” and “I need good data, I need good sound science to make decisions on…”

USDA recognizes that farmers, ranchers, and consumers alike – use data daily, from deciding when to plant, harvest or sell their crops, when to turn out cattle to pasture, or where to buy fresh fruits and vegetables.  This is why it is important that data be made available, open and accessible, to facilitate the best-informed decisions.

Around the world, a movement called the “open data revolution” is under way to make data available for public use.  This movement is expected to generate new insights, drive better decision-making, and enable governments, civil society, and the private sector to better target interventions and programs.

All of this is why the U.S. Government, led by USDA, was a founding partner of the Global Open Data for Agriculture and Nutrition (GODAN) initiative and why it continues to support the advancement of open data for agriculture and nutrition around the world.  Now with over 500 partners, GODAN continues to support the sharing of available, accessible, and usable open data for agriculture and nutrition to help ensure global food security.

If we’re going to feed over 9 billion people by 2050, we need open data policies to make decisions based on facts and evidence. This global perspective will help identify data already available and data gaps that exist, and sharpen the focus on how open data can foster innovation and collaborative research, creating whole new kinds of growth around the world.

The post Open Data Revolution to Fight Global Hunger appeared first on Data.gov.

2013 Tanker Vessel Density from Department of Commerce: https://catalog.data.gov/dataset/2013-tanker-vessel-density

Offshore Wind Resources from the Department of Energy: https://catalog.data.gov/dataset/offshore-wind-resource-10517

BOEM Wind Planning Areas from the Bureau of Ocean Energy Management, Department of the Interior: https://catalog.data.gov/dataset/boem-wind-planning-areas

2013 Fishing Vessel Density off the coast of Oregon.

The post Visualizing ocean traffic on Marine Cadastre appeared first on Data.gov.

Data USA, an online application developed by a team of data scientists at MIT Media Lab and Datawheel, backed by Deloitte is helping Americans visualize demographic and economic data using an open source platform. Where are the Jobs, another online application developed by SymSoft Solutions provides insights on employment trends and salary information by geographic data. The application won first place at the U.S. Department of Labor’s 2011 Occupational Employment Statistics Challenge. Data.gov was the subject of researchers Rashmi Krishnamurthy and Yukika Awazu in their paper, “Liberating data for public value: The case of Data.gov,” International Journal of Information Management (2016). FarmPlenty, recipient of the USDA-Microsoft Innovation Challenge Grand Prize award, uses open government data to helps farmers better analyze U.S. Department of Agriculture data on crops grown within a five mile radius of their farms.

Data.gov’s influence reaches across borders. The Data.gov Help Desk receives numerous requests from U.S. and foreign residents seeking open data insights, technical information, and advice on how to encourage their governments (national, state, local, and tribal) to introduce and improve their open data platforms.

The future of open data is promising because of the collaborative efforts of partners in the private and public sectors. This year, the White House Office of Science and Technology Policy and the Center for Open Data Enterprise launched the Open Data Roundtables to identify open data case studies, learned lessons, and best practices across the Federal government; strengthen a community of technical, legal, and policy experts in support of open data; and support continuity and accelerate the progress of open data work.

Open government data is a team effort, so we ask you to join us and help improve the next year of Data.gov!

Explore

Request

Share

The post 7 Years of Open Data appeared first on Data.gov.

If you’re interested in finding similar data for your ocean area of interest try starting with these datasets on Data.Gov:

Aquaculture Data, Department of Agriculture: https://catalog.data.gov/dataset/aquaculture-data

Critical Habitat Designations, Department of Commerce: https://catalog.data.gov/dataset/critical-habitat-designations-as-of-february-2015

U.S. International Trade in Goods and Services, Department of Commerce: https://catalog.data.gov/dataset/us-international-trade-in-goods-and-services

Northeast Ocean Data Portal Website

The post The Northeast Regional Ocean Council and Maritime Data appeared first on Data.gov.

Link to National Census of Ferry Operators dataset: https://catalog.data.gov/dataset/national-census-of-ferry-operators-data-query-tool-be1e5

The Staten Island Ferry moving through the waters around New York City.

The post Ferrying data to the public through the USDOT National Census of Ferry Operators appeared first on Data.gov.

The 2016 Open Data Roundtables

The Open Data Roundtables were gatherings to identify
case studies, lessons learned, and best practices in open data across the Federal government.

PURPOSE

The Roundtables supported the Federal goal to responsibly unleash government data for the benefit of the American public and maximize the nation’s return on its investment in data.

WHY OPEN DATA?

Open data from government is free, publicly-available data that anyone can use and republish. This important public resource has helped Americans find better value for college, fair housing, and safer medicines. In addition, it helps government agencies operate more efficiently, share information, and engage the citizens they serve.

ABOUT THE 2016 OPEN DATA ROUNDTABLES.

Federal agencies face many obstacles in making their data as accessible and usable as possible. Agencies have been working on these goals and the Roundtables helped identify scalable solutions across the government.

The White House Office of Science and Technology Policy and the Center for Open Data Enterprise hosted four Open Data Roundtables in 2016. The Roundtables were designed to achieve the following goals:

• Identify open data case studies, learned lessons, and best practices across the Federal government;
• Strengthen a community of technical, legal, and policy experts in support of open data; and
• Support continuity and accelerate the progress of open data work.

The Roundtables brought together participants from Federal agencies, academia, the private sector, and nonprofit organizations. The Roundtables included experts on all aspects of open data with technical, policy, and legal backgrounds to identify case studies, lessons learned, and best practices.

ROUNDTABLE TOPICS

FINAL REPORT

The Center for Open Data Enterprise summarized 2016 Open Data Roundtables recommendations from data providers and data users. NOTE: This is not a White House or Federal government document; rather, it is a document produced by civil society to publicly share highlights from the four-part Roundtable series organized in collaboration with the White House.  Download the Final Report

CONTACT

Contact The Center for Open Data Enterprise for more information about the 2016 Open Data Roundtables.

The post Open Data Roundtables appeared first on Data.gov.

Link to dataset: https://catalog.data.gov/dataset/worldwide-historical-hurricane-tracks-from-1848-through-the-previous-hurricane-season

Historical Hurricane Tracks Online Viewer: https://coast.noaa.gov/hurricanes/

150+ years of hurricanes in the Atlantic Ocean

The post Stormy weather: 150+ years of hurricane tracks in the Atlantic Ocean appeared first on Data.gov.

The 2016 Port Performance report used multiple sources, including public datasets featured on Data.Gov. One foundational dataset used in the report is the total commercial tonnage carried on waterways published by the U.S. Army Corps of Engineers – Waterborne Commerce Statistics Center.

Link to dataset: https://catalog.data.gov/dataset/total-tonnage-foreign-and-domestic-of-commodites-carried-on-commercial-waterways .

            Typical infrastructure and cargo flow at a port terminal handling dry bulk cargo, such as coal.

The post By the numbers: port statistics for some of the largest U.S. ports appeared first on Data.gov.

For most users of Data.gov, there should be no impact. For users of Data.gov, Geoplatform.gov and inventory.data.gov with  access to edit web page content, harvest metadata records, or manage metadata records, there are some important changes in the new infrastructure, in particular the integration of Data.gov with OMB MAX for account login authentication, as described in more detail below:

Data.gov (WordPress):

All users who have accounts on Data.gov and signed on previously through http://idp.geoplatform.gov will now use OMB MAX login. If you have an account on Data.gov to modify content on the main Data.gov page or a topic page, you will be sent to OMB MAX upon your first login. You should log in to OMB MAX with your OMB MAX credentials on your first use of the “new” Data.gov. Two-factor authentication (2FA) is now required for Data.gov access, so enable 2FA through the instructions on the OMB MAX log in page. You can add a device by clicking on “Manage SMS 2-Factor Devices” under your profile settings. This is not necessary if you use your PIV/CAC card to log in to MAX; it is already considered 2FA. After you have logged in to OMB MAX, email us at datagov@gsa.gov to let us know that you have completed the initial login. In the email provide the Topic name and access permission (Admin or Editor) required. We will then associate requested permissions to your account. On subsequent logins to Data.gov (through OMB MAX), you should be able to perform the actions you have privileges for on Data.gov without further assistance.

Data.gov (Catalog – CKAN)

Data.gov accounts for managing the Data.gov catalog (CKAN) are mainly for users who need to manage Data.gov harvest sources other than agency data.json files. For federal agencies with established data.json harvest sources for their metadata, there is no need for an account to manage the Data.gov catalog. If you have ad hoc needs, for example to conduct a manual harvest of your agency’s data.json outside of the usual nightly harvest schedule, just contact the Data.gov team at datagov@gsa.gov.

For Data.gov users with accounts to manage the Data.gov catalog (mainly geospatial harvest sources), the Data.gov login that previously used http://idp.geoplatform.gov will now use OMB MAX login. You will be sent to OMB MAX on your first login, you should log in to OMB MAX using your OMB MAX credentials on your first use of the “new” Data.gov catalog. Two-factor authentication (2FA) is now required for Data.gov access, so enable 2FA through the instructions on the OMB MAX log in page. You can add a device by clicking on “Manage SMS 2-Factor Devices” under your profile settings. This is not necessary if you use your PIV/CAC card to log in to MAX; it is already considered 2FA. After you have logged in to OMB MAX, email us at datagov@gsa.gov to let us know that you have completed the initial login. In the email provide the organization name and access permission (Admin or Editor) that is required. We will then associate requested permissions to your account. On subsequent logins to Data.gov (through OMB MAX), you should be able to perform the actions you have privileges for on the Data.gov catalog without further assistance.

If you have an urgent need for a change to your harvest source/schedule during the first several days post migration, please contact datagov@gsa.gov and we will make the changes for you.

Geoplatform.gov users (Website only)

The Data.gov migration and authentication changes only affect Geoplatform.gov users who manage harvest sources for the Geoplatform.gov data catalog through http://catalog.data.gov.  If you have permissions to use Geoplatform.gov for any other purpose, there is no change to your login through http://idp.geoplatform.gov. If you have questions regarding Geoplatform.gov, contact servicedesk@geoplatform.gov.

If you manage harvest sources for the data catalog at Geoplatform.gov, please refer to the instructions for Data.gov catalog users above and follow the same instructions for logging in through OMB MAX. If you don’t have access to OMB MAX- see the section below on obtaining access.

Inventory.data.gov

Inventory.data.gov is the non-public system provided by the Data.gov team, mainly to assist federal agencies in managing their metadata for inclusion in the Data.gov catalog. Agency users have access to and are able to manage their own agency’s metadata only. Logins to use inventory.data.gov are also now integrated with OMB MAX. Log in on inventory.data.gov will now take you to OMB MAX login. You should log in to OMB MAX using your OMB MAX credentials on your first use of inventory.data.gov post migration. Two-factor authentication (2FA) is now required, so enable 2FA through the instructions on the OMB MAX log in page. You can add a device by clicking on “Manage SMS 2-Factor Devices” under your profile settings. This is not necessary if you use your PIV/CAC card to log in to MAX; it is already considered 2FA. After you have logged in to OMB MAX, email us at datagov@gsa.gov to let us know you have completed the initial login, in the email provide the organization name and access permission (Admin or Editor) required. We will then associate requested permissions to your account. On subsequent logins to inventory.data.gov (through OMB MAX), you should be able to use inventory.data.gov to manage metadata as previously without further assistance.

Don’t have an OMB MAX Account?

If you are a federal government employee or a federal government contractor with a .gov email, you can self register for an OMB MAX account.

For non-federal users (mainly managing non-federal harvest sources for Geoplatform.gov through http://catalog.data.gov ), you can obtain OMB MAX access through a federal sponsorship process. This is a relatively new process for OMB MAX that we will be working on with the federal sponsors on the Geoplatform.gov side.  We will be refining this process in the next few weeks so that federal contacts can sponsor/add non-federal users for OMB MAX more easily. Please contact servicedesk@geoplatform.gov.

Thanks for your patience as we complete the Data.gov migration questions and for your understanding on any initial glitches. Contact us at datagov@gsa.gov with any other questions.

The post Data.gov Infrastructure Migration – Notice to Federal and Non-Federal Users appeared first on Data.gov.

The post A Maritime Topic for a Maritime Nation appeared first on Data.gov.

Link to dataset: http://catalog.data.gov/dataset/accident-statistics  

The post Trends in Recreational Boating Safety, U.S. Coast Guard Recreational Boating Safety Statistics Tool appeared first on Data.gov.

Link to dataset: http://catalog.data.gov/dataset/piers-shipping-information-u-s-waterborne-foreign-trade-by-u-s-custom-districts-c6440

The post A Decade of Shipment Tonnage and Value Summaries appeared first on Data.gov.

This Grain Transportation Report: Rail Deliveries to Port dataset from the USDA provides the total number of rail carloads used in weekly grain shipments traveling to ports in five different regions of the country starting in 1994.  The five regions are Atlantic, East Gulf, Mississippi River, Texas Gulf, and Pacific.  In 2008 the dataset started recording cross border rail shipments in to Mexico.  Included with each record is start-of-week date, end-of-week date, and number week of year (1-52).  For users interested in learning about intermodal freight flows, agricultural exports, and how the marine transportation system supports American farmers this dataset is a great resource.

A glossary of terms used in the file is available from USDA at: https://www.ams.usda.gov/sites/default/files/media/GTRGlossaryofTerms.pdf

View Dataset link:  http://catalog.data.gov/dataset/grain-transportation-report-table-3

Reference: https://www.ams.usda.gov/services/transportation-analysis/gtr

The post Harvesting Grain Data appeared first on Data.gov.

Data.gov invites you to share your open data stories as you explore or download specific open government datasets. Doing so will provide feedback to government agencies about which datasets are in high demand and which ones need improvement. It will also help the Data.gov team curate open data topics and special features- including coverage of open data events and hackathons.

Share your open data story here

The post Your Open Data Story appeared first on Data.gov.

Click here for the data.

Mapping applications like PolicyMap are incorporating USDA’s rural housing data and overlaying them with other indicators.

The post Introducing New Rural Housing Data from USDA appeared first on Data.gov.

Surging Seas uses government data to project sea level rise scenarios for the coastal United States.

The post Climate Data in Action appeared first on Data.gov.

The post Release of Infrastructure and Geographic Map Data for Climate-Preparedness appeared first on Data.gov.

Using the data, tools, and maps tabs, you can find more datasets related to coastal flooding risks to communities. If you are an entrepreneur or innovator looking for a problem to solve, visit the challenges.

Here you can find datasets related to coastal flooding risks to communities. In the coming months, we’ll feature datasets, services, and tools related to:

• Climate change and human health
• Vulnerability of food systems to climate change
• Vulnerability of the energy supply to climate change
• Vulnerability of ecosystems
• …and more

Your input will help make Climate.Data.gov as useful as possible—please share your feedback.

The post What Kind of Climate Data Can I Find? appeared first on Data.gov.

Check out the data catalog to browse relevant datasets. If you are looking for a streamlined list, the resources page features datasets and services on coastal vulnerability.

Looking for information to help you and your community plan for coastal flooding and sea level rise? Please review our list of tools, which will grow over time.

If you are a problem-solver or entrepreneur who wants to take on a big challenge to help communities and citizens be more aware and prepared for climate change, check out the challenges.

The post Welcome to Climate.Data.gov! appeared first on Data.gov.

The post NOAA and NASA Launch Coastal Vulnerability Innovation Challenge appeared first on Data.gov.

The post NASA launches Earth Science Challenges appeared first on Data.gov.

The post Infrastructure and Geographic Map Data for Climate-Preparedness appeared first on Data.gov.

These data are also being made available via mapping services on Geoplatform.gov. The resources provided here can be used to answer a number of relevant questions, such as:

FOOD PRODUCTION

1. How will crop yield and production of other food products be affected by changes in climate?
2. What types of food and locations will be most vulnerable to changes in climate?
3. How will changes in climate affect the types of food that can be grown in a location?
4. Under what conditions (i.e., for what types of food and in what locations) will indirect effects of climate change (e.g., pests, pathogens, invasive species, and fire) overwhelm direct effects of increasing temperature and/or changes in precipitation, surface water, or ground water?

FOOD DISTRIBUTION

1. What parts of the food supply chain will be most vulnerable to climate change?
2. How will processing, storage, and transportation need to be modified to continue to supply safe food products under a changing climate?

FOOD SAFETY AND NUTRITION

1. How will climate change impact foodborne illness that includes chemical as well as microbial sources?
2. How will climate change affect the nutrient content of staple foods?
3. How will the cost of a balanced diet be impacted under a changing climate?

INTERNATIONAL FOOD SECURITY

1. What countries are most vulnerable to climate change that affects food yield, quality, and availability?
2. What segment of the population will be most vulnerable to changes in food prices?

The post Launch of Food Resilience theme of Climate.Data.Gov appeared first on Data.gov.

A new Data.gov community (Ecosystems.data.gov) has been launched to serve as an entry point for access to the U.S. Ecoinformatics-based Open Resources and Machine Accessibility (EcoINFORMA) initiative, including the EcoINFORMA data resource hubs (Biodiversity, Ecosystem Services, and Land Cover Dynamics), the EcoINFORMA map viewer for visualizing and integrating geospatial data, and access to additional biodiversity, ecosystems, ecosystem services and environmental data resources for use by researchers, decision-makers and the public. EcoINFORMA is an initiative that responds to recommendations from the President’s Council of Advisors on Science and Technology (PCAST) 2011 report on “Sustaining Environmental Capital: Protecting Society and the Economy”.

The post Ecosystems.data.gov and EcoINFORMA appeared first on Data.gov.

The post Ecosystem Vulnerability Launch appeared first on Data.gov.

• Help communities and natural resource managers determine if they are currently at risk from wildfires and if they will be impacted in the future due to wildfires becoming more prevalent and severe;
• Provide information to the public on their sources of water and their sensitivities to climate change;
• Aid in the public understanding of the role that ecosystems play in mitigating rising carbon dioxide levels due to their absorbing and storing of carbon, as well as how land management activities may influence storage capabilities;
• Identify the potential impacts of climate change on rare and endangered species, iconic species, and ecosystems;
• Identify which invasive species may threaten specific locations and their impacts on local communities and their economies. This effort will contribute to early detection, rapid response activities.

The post Launch of Ecosystem-Vulnerability theme of Climate.Data.Gov appeared first on Data.gov.

(April 7th 2015) The U.S. Government has released a collection of datasets to help individuals and communities plan for the impacts of climate change on the public’s health. These resources can help answer a number of relevant questions, including:

• In what ways does the changing climate affect public health where I live?
• What risk factors make individuals or communities more vulnerable to climate-related health effects?
• How can public health agencies, communities, and individuals plan for uncertain future conditions?

The post Launch of Health theme of Climate.Data.Gov appeared first on Data.gov.

In a continued effort to provide key datasets that support the process of developing resilience to climate change, five additional datasets are now available in the Ecosystem Vulnerability theme.

Those include:

• California Ocean Uses Atlas

• Environmental Sensitivity Index (ESI) Threatened and Endangered Species REST Services

• US Forest Service Forests To Faucets

• National Wildlife Chemical Effects Database

• Threatened and Endangered Terrestrial Animal Species Richness

The post Ecosystem Vulnerability Theme Adds Five New Datasets appeared first on Data.gov.

U2U is composed of 50 faculty, staff and students from nine universities who specialize in applied climatology, crop modeling, agronomy, cybertechnology, agricultural economics and other social sciences. It is funded by the U.S. Department of Agriculture’s National Institute of Food and Agriculture.

The post A multi-university initiative makes global climate information more accesible to farmers appeared first on Data.gov.

These data are also being made available via mapping services on Geoplatform.gov. The resources provided here can be used to explore and develop insights for a number of relevant questions, such as:

1) How are fundamental energy resources impacted by climate?

2) How might changes in climate and natural resource availability impact energy conversion infrastructure and processes?

3) How might climate impact energy transmission and distribution systems?

4) How might energy demands be impacted by climate change, including heating and cooling but also energy losses and energy used for adaptation by other sectors?

5) What capacity do we currently have to adapt energy systems, and how might technology solutions, systems designs, and operational changes improve energy system resilience for climate change?

6) How might climate change impact energy infrastructure and its interactions with networked and interconnected infrastructure systems?

The post Launch of Energy & Infrastructure Resilience theme of Climate.Data.Gov appeared first on Data.gov.

The Innovation Challenge was formally launched on July 27^th at a conference of the Agricultural & Applied Economics Association in San Francisco. Challenge participants have 3 months to create their applications, with a top prize of $25,000 going to the most creative application that best exploits USDA data sets that are now being hosted on Microsoft Azure, Microsoft’s cloud computing platform.

Entrants are invited to develop and publish new applications and tools that can help users analyze multiple sources of information, including key USDA data sets. In addition, Microsoft is granting cloud computing awards to aid university researchers and students that are looking to take part in the challenge.  Challenge winners will be announced in December 2015.

Full details of the challenge can be found at >http://usdaapps.challengepost.com<.

The post Microsoft Launches “Innovation Challenge” around Food Resilience appeared first on Data.gov.

The National Academy of Sciences Building
Washington, DC

The Transportation Research Board will host a conference September 16-18, 2015 to provide transportation professionals with information about emerging best practices and research results on how to adapt surface transportation networks to the potential impacts of climate change and extreme weather events. The conference will examine efforts to mainstream consideration of climate change and extreme weather resilience in all aspects of the transportation sector, including planning and programming, capital improvements, and operations and maintenance.

The post First International Conference on Surface Transportation System Resilience to Climate Change and Extreme Weather Events appeared first on Data.gov.

Explore Data USA: http://datausa.io

More from Data USA: http://bcove.me/80uv9you

The post Data USA: Visualizing Government Data appeared first on Data.gov.

Below is an abstract of “Liberating data for public value: The case of Data.gov,” International Journal of Information Management (2016).

Abstract

Public agencies around the globe are liberating their data. Drawing on a case of Data.gov, we outline the challenges and opportunities that lie ahead for the liberation of public data. Data.gov is an online portal that provides open access to datasets generated by US public agencies and countries around the world in a machine-readable format. By discussing the challenges and opportunities faced by Data.gov, we provide several lessons that can inform research and practice. We suggest that providing access to open data in itself does not spur innovation. Specifically, we claim that public agencies need to spend resources to improve the capacities of their organizations to move toward ‘open data by default’; develop capacities of community to use data to solve problems; and think critically about the unintended consequences of providing access to public data. We also suggest that public agencies need better metrics to evaluate the success of open-data efforts in achieving its goals.

Rashmi Krishnamurthy, School of Public Affairs, Arizona State University, Phoenix, AZ,USA

Yukika Awazu, The Institute for Knowledge and Innovation South-East Asia, Bangkok University, Bangkok, Thailand

http://www.sciencedirect.com/science/article/pii/S0268401216301098

The post Liberating data for public value: The case of Data.gov appeared first on Data.gov.

FarmPlenty helps farmers better analyze U.S. Department of Agriculture (USDA) open data on crops grown within a five mile radius of their farms. Inspired by his conversations with farmers on the challenges faced in predicting crop yields and consumer demand, founder George Lee built his application as part of the USDA-Microsoft Innovation Challenge. His Grand Prize-awarded application is supported by the USDA’s National Agricultural Statistics Service (NASS) CropScape and Quickstats APIs.

Application

FlockPlenty – Chicken Egg Tracker

The post FarmPlenty appeared first on Data.gov.

These datasets include:

• NOAA Climate Data Record (CDR) of Sea Surface Temperature -WHOI, Version 1.0
• U.S. Monthly Extremes
• Global Historical Climatology Network – Monthly (GHCN-M) Version 3
• African Easterly Wave Climatology Version 1
• NOAA Climate Data Record (CDR) of Daily Outgoing Longwave Radiation (OLR), Version 1.2
• NOAA Climate Data Record (CDR) of Monthly Outgoing Longwave Radiation (OLR), Version 2.2-1
• Global Surface Summary of the Day – GSOD
• Monthly Summaries of the Global Historical Climatology Network – Daily (GHCN-D)
• International Surface Temperature Initiative (ISTI) Global Land Surface Temperature Databank – Stage 1 Monthly
• International Surface Temperature Initiative (ISTI) Global Land Surface Temperature Databank – Stage 2 Monthly
• International Surface Temperature Initiative (ISTI) Global Land Surface Temperature Databank – Stage 3 Monthly
• International Surface Temperature Initiative (ISTI) Global Land Surface Temperature Databank – Stage 1 Daily
• International Surface Temperature Initiative (ISTI) Global Land Surface Temperature Databank – Stage 2 Daily

The post Coastal Flooding Theme Adds Thirteen New Datasets appeared first on Data.gov.

Those include:

• USDA National Nutrient Database for Standard Reference
• Germplasm Resources Information Network
• Farm Program Atlas
• Climate Prediction Center (CPC) U.S. Seasonal Drought Outlook (SDO)
• 2012 Census of Agriculture
• World Agricultural Production
• National Animal Health Monitoring System (NAHMS)

The post Food Resilience Theme Adds Seven Datasets appeared first on Data.gov.

• How are human and natural components of the hydrologic cycle changing?
• How can communities and water managers plan for uncertain future conditions?
• How will changing water resources affect food, energy, ecosystems, and human health?

The post Launch of Water theme of Climate.Data.Gov appeared first on Data.gov.

Those include:

• Global Lake Temperature Collaboration
• Forced Evaporation from Water Surface
• Groundwater depletion in the United States (1900−2008)
• California drought, visualized with open data
• Federal Support Toolbox

The post Water Theme Adds New Content appeared first on Data.gov.

These datasets include:

• Quality Controlled Local Climatological Data (QCLCD) Publication
• U.S. Hourly Climate Normals (1981-2010)
• U.S. Daily Climate Normals (1981-2010)
• U.S. Monthly Climate Normals (1981-2010)
• U.S. Annual/Seasonal Climate Normals (1981-2010)
• NCDC Storm Events Database
• Annual Climatological Summaries
• NOAA’s Climate Divisional Database
• U.S. Climate Reference Network (USCRN) Hourly Products
• U.S. Climate Reference Network (USCRN) Daily Products
• U.S. Climate Reference Network (USCRN) Monthly Products

The post Ecosystem Vulnerability Theme Adds Eleven New Datasets appeared first on Data.gov.

These datasets include:

• NOAA Optimum Interpolation 1/4 Degree Daily Sea Surface Temperature (OISST) Analysis, Version 2
• AVHRR Pathfinder Version 5.2 Level 3 Collated (L3C) Global 4km Sea Surface Temperature (SST) Climate Data
  Record (CDR) for 1981-2010
• NOAA Climate Data Record (CDR) of Gridded Satellite Data from ISCCP B1 (GridSat-B1) 11 micron Brightness Temperature, Version 2
• NCDC Storm Events Database
• Coastal Economic Trends for Coastal Geographies
• Demographic Trends (1970-2010) for Coastal Geographies
• FEMA HAZUS Critical Facilities for Coastal Geographies
• Time-Series Data for Self-Employed Economic Activity Dependent on the Ocean and Great Lakes Economy for Counties, States, and the Nation between 2005 and 2012
• Time-Series Data on the Ocean and Great Lakes Economy for Counties, States, and the Nation between 2005 and 2012 (Sector and Industry Level)
• Time-Series Data on the Ocean and Great Lakes Economy for Counties, States, and the Nation between 2005 and 2012 (Sector Level)
• Social Vulnerability Index (SoVI) for the U.S. Coastal States based on the 2010 Census Tracts

The post Coastal Flooding Theme Adds Eleven New Datasets appeared first on Data.gov.

• USGS report: “Data Categories for Marine Planning” http://dx.doi.org/10.3133/ofr20151046
• USGS Information for Marine Planners and Resource Managers http://marine.usgs.gov/marineresourceinfo/

The post New Report – Data Categories for Marine Planning appeared first on Data.gov.

In February 2015, Data.gov was recognized for its technological innovations by placing in the top eight of more than 100 submissions at the ACT-IAC Innovation Awards. Data.gov was honored for its innovative transformation in the last year from a central metadata catalog (to which agencies submitted datasets one-by-one) to a data harvesting hub (where agency datasets are now automatically gathered on a daily basis). This provides a more structured and efficient way to share and discover federal open data and ensures that users can count on Data.gov as an up-to-date source for such data.

Data.gov uses open source technology to power the website (WordPress) and the data catalog is developed using CKAN (Comprehensive Knowledge Archive Network). Using open source technology allows developers from around the globe to give input, and for others to re-use the code. This technology also results in greater integration with state, city, and county catalogs. Data.gov currently syndicates 37 local government data catalogs and more are added every month.

Data.gov is bringing transparency to the Federal Government’s open data efforts. Through the Project Open Data dashboard, anyone can see the performance of federal agencies in meeting the Open Data Policy requirements. The public can also weigh in on issues, and see how agencies are responding to recommendations, creating greater accountability as well.

Data.gov has launched several new features this year. To improve public engagement on open data, Data.gov launched its open, API-driven, customer service platform, the Data.gov Help Desk, which allows for the public to submit issues, improvements, and requests for government data. The status of all data issues can be publicly tracked.

Another new feature is the integration of data visualization tools, giving Data.gov users the option to open datasets using third party tools like plot.ly and CartoDB.

In the year ahead, Data.gov will continue to work with agency partners to take open data to another level. We will expand the scope of the catalog and improve the quality of metadata. We will develop and launch additional tools to enable agencies to release even more of their data, so that government data can continue to inform citizens and improve their daily lives.

From the example set by the U.S. and Data.gov in 2009, there are now over 75 national open data sites. Data.gov will continue to set the lead unleashing the power of open data around the world.

1 – A note on metrics: There are many nuances associated with the number of datasets on Data.gov since it syndicates many types of data from many different sources. Some datasets are grouped into collections and each dataset can include multiple resources. In total, there are currently over 4 million resources (such as individual download links) cataloged on Data.gov.  Since the first milestone of the Federal open data policy, there have been over 50,000 Federal datasets updated on the site using the metadata management process established by the policy, but many local governments have used the same metadata standard to syndicate to Data.gov as well. In total there are currently over 98,000 Federal datasets currently on Data.gov and over 30,000 from local government sources including over 10,000 using the same metadata standard as the Federal government.

The post Data.gov Turns Six! appeared first on Data.gov.

In coordination with The White House Climate Data Initiative,
the Federal Geographic Data Committee, and the National Food Interoperability Experiment,
the AWRA Technology Committee is supporting a national discussion titled
“The Open Water Data Initiative – Unifying Water Data for the Nation.”

For more information visit http://awra.org/meetings/Annual2014/lightning.html

The post 2014 AWRA Annual Water Resources Conference appeared first on Data.gov.

New tools are making it easier to submit complaints, comments, and petitions to your government online and through mobile applications. Data.gov is proud to announce the launch of the Data.gov Help Desk, a new customer service platform powered by Open311 that will let the public submit issues, improvements, and requests for government data. You’ll now find these options on our Contact page, but soon they will be integrated throughout the site. This is an open, API-driven platform for the public to voice their needs and track the status of data issues. This initial release is just the beginning, but we look forward to hearing your feedback here and on GitHub to make the Help Desk feature even better.

Background

Since the beginning, Data.gov has striven to engage with the public to help software developers, researchers, and other data users find and understand the data they need. Because Data.gov is a small team, we can’t always answer questions as quickly as we would like to. Additionally, these questions are often best answered by specific government agencies and domain experts. A few years ago, we helped establish the Open Data Stack Exchange as a community-run resource for questions and answers about all kinds of open data, not even just government data. The Open Data Stack Exchange is a great resource which we continue to support, but there are many instances when it would be better to integrate these interactions more tightly with data publishers and specific datasets. For example, when someone is looking for a specific government dataset or wants to report a problem or suggest an improvement with an existing dataset, we’d like to make sure that the request makes it to the right agency and that data issues and improvements are tied to their dataset listings on Data.gov.

As part of the Federal Open Data Policy, agencies have been asked to provide a feedback mechanism to engage with their users and prioritize datasets for release. While some agencies like the U.S. Census have well established platforms for engaging with their users, many agencies have been limited in providing more than a basic contact form to facilitate feedback. Similarly, Data.gov hasn’t had a sophisticated process for responding to data inquiries. In the past year, Data.gov has made heavy use of GitHub issues for publicly tracking problems and feature requests for the Data.gov website and occasionally data-specific issues come through as well, but we’ve been in need of a separate process oriented around data to properly delegate to the right folks in government.

Open Feedback for Open Data

Fortunately, there’s already a great model for managing citizen requests in a simple way across the complexity of government. The model of a consolidated contact center and the 311 phone number in many cities has had a significant impact on the ability for people to request services and information from their government and track the progress of their request. More recently the Open311 specification has made those interactions into more of a public forum and standardized the process with a common API that can be used across many different applications and governments. To date, Open311 has mostly been limited to local government, but the White House has taken note in the past.

Since the Open311 standard already provides the foundation for a common data driven customer service platform, the Data.gov Help Desk has been built on top of it.

Open for Integration

Working with the Open311 standard has meant that the the Data.gov Help Desk has taken an API-first approach which means that all of our tools have been developed around the same API that’s available to outside developers (also known as “dogfooding”). In the near future we will make this API available to a wider audience outside of government, following the same precedent as the White House We The People Petitions API, but we will also be working closely with agencies across government so that we can use standardized APIs to integrate with their own customer service platforms and build on top of initiatives like Demand Driven Open Data from HHS. Hopefully we can establish a precedent for a federated standardized API across government in much the same way the Project Open Data schema has standardized metadata across both Federal agencies and local governments.

This initial release of the Help Desk is just a starting point and there will be many opportunities to integrate it across Data.gov and expand its functionality. The most immediate next step will be to incorporate a Help Desk button and list existing issues around a dataset directly on the dataset page. That integration will come very soon. We’re also excited about the potential for using this along with more unstructured conversations such as those using tools like Discourse.

The Data.gov Help Desk is being developed in the open on Github. The primary place where you can provide feedback on the feature is issue #422 in the main Data.gov repository, but other components have their own repositories: Open311 Simple CRM, and Open311-WP. With the GitHub Open311 Adapter, we’re also looking at the possibility of using GitHub Issues itself as a backend that can be served with this same API standard. We’re excited to build this not just as a platform, but as part of a broader ecosystem for engagement around data. We look forward to your participation.

The post Announcing the Data.gov Help Desk appeared first on Data.gov.

New tools are making it easier to visualize and analyze data at the click of a button. Data.gov is proud to announce integration with two of these tools — Plotly and CartoDB — making it possible to create graphs and dynamic maps with open government data in one click.

On supported datasets, a new “Open With” menu will appear giving you the option to open the dataset directly with these third party services. No need to download and upload, just click the link and go.

Plotly

Plotly converts datasets into collaborative graphs and data visualizations for the web. With Plotly you can create and customize a variety of interactive visualizations: bar, line, and scatter graphs, heat maps, area graphs, bubble chart, histograms, linear regressions, box plots, 3D graphs, and more. More advanced users can crunch data with any custom function, (including derivatives, integrals, inferential or descriptive basic statistics, chi-squared test, T-test, and ANOVA) and create simulations with Python scripts. To view existing data visualizations with Plotly, you can explore their graphing news feed.

CartoDB

CartoDB converts geospatial datasets into dynamic customizable maps for the web in seconds. Once the map is automatically created users can filter and refine the visualized data to gain insights and tell stories. Users can customize the appearance of the maps through a variety of beautiful themes and advanced users can leverage CartoCSS, a cascading style sheet language for maps. To see the potential of CartoDB, their website has a gallery of maps using government data.

Both Plotly and CartoDB make it easy to share and edit your work privately, with your co-workers, or the world. There is so much to learn from open government data and we are happy to feature tools like Plotly and CartoDB that make it easier for everyone to become a data scientist. If you create a great data visualization with Data.gov data please let us know!

With More to Come

The integration with Plotly and CartoDB is made possible with nothing more than simple website links and Data.gov will be expanding these options to include other third party services in the future.

The “Open With” menu is currently made available for smaller CSV and XLS files and will not appear when these files are not verified as a functioning CSV or XLS file. Other file formats are also supported, such as KML and zipped Shapefiles for use with CartoDB, but currently Data.gov does not validate these file formats before providing the link. In some cases, the format specified in dataset metadata on Data.gov does not accurately reflect the linked file. If you come across a dataset that is labeled as a CSV or XLS file and the “Open With” menu is not present it is either because the file is too large or because the link does not point directly to the CSV or XLS file.

Data.gov will continue to work with data publishers to update these listings and ensure that the “Open With” menu works for more datasets and connects with even more services.

If you have feedback about the “Open With” menu, please contact us or leave a comment here or on GitHub. We look forward to seeing what you create as you open data with new insight.

The post Open with Apps appeared first on Data.gov.

Data.gov is the central clearinghouse for federal open data, including hosting the Public Data Listings required under the 2013 Federal Open Data Policy, but Data.gov also hosts state, local, and tribal government sources voluntarily. As of this writing Data.gov has the open data catalogs of 10 cities, 3 counties, and 8 states open data catalogs piped to Data.gov, in addition to many local geospatial data sources, but we know there are many more open data catalogs out there!

Getting your local government open data plugged into Data.gov is likely easier than you think. We recently outlined the technical process of how data gets harvested onto Data.gov in detail with our “How to Get Your Open Data on Data.gov” guide. The gist of which is: Data.gov harvests standard metadata and there a slightly different requirements for federal data, federal geospatial data, and non-federal data. For state, county, and city data Data.gov simply needs two things from a government contact:

1. A data.json file (this is likely built into your open data catalog, but an open data catalog is not required, see www.data.gov/faq).
2. A link to your local open data catalog’s Terms of Use or Data Policy.

With those two items, Data.gov can set up nightly harvests that will automatically update Data.gov with your local data.

Local governments should get their data on Data.gov first and foremost because it will help make local open data more discoverable, but also because it supports a minimal metadata standard across local open data efforts, can help identify emergent data trends and standards (like the US City Open Data Census) across local governments, and because it shows the international open data community how active the United States is!

To all local government officials, we would love to hear from you and connect your government data to Data.gov as soon as possible. As always, If you have any questions, you can reach out to datagov@gsa.gov.

Add your local government open data today by visiting https://www.data.gov/local/add.

The post Get your Local Government on Data.gov appeared first on Data.gov.

The post National Institute for Food and Agriculture’s Research, Education, and Economics Information System appeared first on Data.gov.

–Lynn Overmann, Senior Advisor to the U.S. CTO

The post County Business Patterns appeared first on Data.gov.

Through openFDA, developers and researchers will have easy access to high-value FDA public data through RESTful APIs and structured file downloads. In short, our goal is to make it simple for an application, mobile, or web developer, or all stripes of researchers, to use data from FDA in their work. We’ve done an extensive amount of research both internally and with potential external developers to identify which datasets are both in demand and have a high barrier of entry. As a result, our initial pilot project will cover a number of datasets from various areas within FDA, defined into three broad focus areas: Adverse Events, Product Recalls, and Product Labeling. These API’s won’t have one-on-one matching to FDA’s internal data organizational structure; rather, we intend to abstract on top of a myriad of datasets and provide appropriate metadata and identifiers when possible. Of course, we’ll always make the raw source data available for people who prefer to work that way (and it’s good to mention that we also will not be releasing any data that could potentially be used to identify individuals or other private information).

The openFDA initiative is one part of the larger Office of Informatics and Technology Innovation roadmap. As part of my role as FDA’s Chief Health Informatics Officer, I’m working to lead efforts to move FDA in to a cutting edge technology organization. You’ll be hearing more about our other initiatives, including Cloud Computing, High Performance Computing, Next Generation Sequencing, and mobile-first deployment in the near future.

As we work towards a release of openFDA we’ll begin to share more about our work and how you can get involved. In the meantime, I suggest you sign up for our listserv (on our home page) to get the latest updates on the project. You can also reach our team at open@fda.hhs.gov if there is a unique partnership opportunity or other collaboration you wish to discuss.

Dr. Taha Kass-Hout is the Chief Health Informatics Officer of FDA

taha@fda.hhs.gov | @DrTaha_FDA

The post Introducing OpenFDA from the Food and Drug Administration appeared first on Data.gov.

Water column sonar data are collected on NOAA fishery survey vessels and academic and international fleets, and are used to assess the physical and biological characteristics of the ocean. Primary uses include 3-D mapping of fish schools and other mid-water marine organisms, assessing biological abundance, species identification, and habitat characterization. These data are also useful for evaluating underwater gas seeps, remotely monitoring undersea oil spills, and bathymetry.

NGDC is working with scientists at NMFS and the Joint Hydrographic Center to ensure the long-term preservation and world-wide dissemination of these data.

The post Water Column Sonar Data Collection appeared first on Data.gov.

Presenter(s): Mimi D’Iorio (National Oceanic and Atmospheric
Administration), Laura McKay (Virginia Coastal Zone Management Program),
Nick Meade (Virginia Coastal Zone Management Program), Jeanne Herb (Rutgers
University)

Description: Understanding human uses of the ocean is essential to successful marine
planning. Unfortunately, spatial data on ocean uses are often limited and
difficult to capture consistently over large areas. Participatory
geographic information system (PGIS) processes provide interactive ways to
capture local knowledge and ocean use patterns through specialized GIS
mapping tools. NOAA has been working with partners all over the country to
apply this method at local and regional scales. This webinar will be a
panel discussion with NOAA and its partners in the mid-Atlantic region
focused on the process, data, and lessons learned.

In this webinar, participants will:

– Learn about the NOAA PGIS method for capturing ocean use data and
where it has been applied;
– Hear insights from the mid-Atlantic team that has taken the process
from one state to the entire region for marine planning applications; and
– Gain a better understanding about the data, products, and often
unexpected outcomes of PGIS projects.

Registration Link

The post Webinar: “Using Participatory GIS to Map Ocean Uses in the Mid-Atlantic” appeared first on Data.gov.

Wednesday, June 4, 2014

2:00 – 3:00pm EST

Presenter(s): Jeff Adkins and Linwood Pendleton

A lot of people’s livelihoods depend on the resources of the oceans and Great Lakes. Economics: National Ocean Watch (ENOW) is the only nationally consistent data set that is focused on monitoring the ocean and Great Lakes economy. These time-series data report the establishments, employment, wages, and gross domestic product for six ocean-dependent sectors. ENOW provides data for about 400 counties, 30 coastal states, 8 regions, and the nation. This webinar will be an interactive dialogue between Duke University Nicholas Institute scholar Linwood Pendleton and NOAA economist Jeff Adkins. They will be sharing highlights and data development methods, related international efforts, and ways in which the data can be used to improve coastal and ocean management.

In this webinar, participants will:

• Get an overview of ENOW data and methods
• Understand the value of the U.S. ocean and Great Lakes economy
• Learn how to apply data for coastal and ocean management

Registration Link – LINK

The post Webinar: “Monitoring the US Ocean and Great Lakes Economy” appeared first on Data.gov.

Monday, June 2, 2014, 3 to 4p.m. Eastern

Presenters: Daniel Martin, NOAA Coastal Services Center; Stephen Creed,
Bureau of Ocean Energy Management; Kyle Ward, NOAA Office of Coast Survey;
Andrea Dransfield, Channel Islands National Marine Sanctuary

Vessel traffic data, or Automatic Identification System (AIS) data, are
collected by the U.S. Coast Guard through an onboard navigation safety
device that transmits and monitors the location and characteristics of
large vessels in real time. The MarineCadastre.gov project team has worked
with the Coast Guard and NOAA’s Office of Coast Survey to repurpose and
make available some of the most important data for use in ocean planning
applications. This webinar will provide an overview of the data available
and examples of how the MarineCadastre.gov AIS data are being used,
including scoping wind planning areas, evaluating ship anchorage areas, and
analyzing changes to shipping lanes through a national marine sanctuary.

In this webinar, participants will

• Learn about the variety of vessel-tracking data products and tools and how to access them via MarineCadastre.gov
• Gain a better understanding of how the products can be applied to a range of ocean-planning applications

Registration Link:
http://noaacsc.adobeconnect.com/june022014/event/registration.html

The post Webinar: Vessel Tracking Data Used for Ocean Planning appeared first on Data.gov.

The post NOAA’s Ship Tracker appeared first on Data.gov.

The post Virginia Atlantic Coast Recreational Use appeared first on Data.gov.

While Data.gov lists its holdings at almost 400,000 datasets, the fact is, Data.gov is the link to millions of public datasets.

Many Federal agencies post their data collections on Data.gov as single entities. In addition, Data.gov is host to tools, widgets and gadgets that each accesses thousands of datasets.

These collections run the gamut of topics. For example:

•  2011 Code of Federal Regulations in XML. It’s not surprising that the codification of the general and permanent rules published in the Federal Register by the Federal agencies would reach 100 MB in size.
• Bureau of Land Management, General Land Office (GLO)  Records System. About 10 million records. The GLO holds U.S. land title information images, including patent images, survey plats, field notes and master title plats.
• EPA Enforcement and Compliance History Online (ECHO) Exporter Data. ECHO shares data related to water, air, waste, and drinking water regulations, along with information on facilities violating the Clean Water Act and tools that compare water quality trends and generate interactive maps that link to state dashboards.
• Farmers Market Geographic Data. Longitude and latitude, state, address, name, and zip code of nearly 8,000 Farmers Markets in the United States.
• Federal Advisory Committee Act (FACA) Database. Records of more than 1,000 government advisory committees from 1972 to the present.
• NASA Image Exchange (NIX). A collection of more than 500,000 multimedia records from NASA’s missions and projects.
• Patent Grant Bibliographic Text (1976-present). With 4,000 patents being issued each week, this database contains about 10 million records. Like many agencies, the U.S. Patent and Trademark Office is a wealth of data. We’ve also got the Patent Classification Files since 1790.

Data.gov is not about the numbers but the quality of our holdings. Still, it’s important to realize that a vast storehouse of data can be found behind just one little click.

Sally Ruth Bourrie of Phase One Consulting Group supports Outreach and Communications at Data.gov.

The post One Click and We’ll Give You Millions appeared first on Data.gov.

The post ERS Food Access Research Atlas API appeared first on Data.gov.

The post Maritime Limits and Boundaries of the United States appeared first on Data.gov.

The post National Agriculture Statistics Service QuickStats appeared first on Data.gov.

The new catalog consolidates all of Data.gov’s holdings in one easy-to-use catalog. It features a number of improvements, such as an improved search that helps you find all the datasets for a particular location, such as your zip code, better sorting and tagging of datasets, and improved metadata. Data.gov now has one unified data catalog based on an open source standard that will make it easier to federate with other federal agency catalogs, as well as those of states, cities, and counties. The new CKAN catalog will also enable the implementation of the Open Data Policy, as it will easily harvest the data inventories that federal agencies will be creating under the policy. The result will be a user-friendly, comprehensive catalog that will allow citizens, developers and others to fully take advantage of the vast array of federal data that affects the daily lives of citizens. 

The post Data.gov CKAN Catalog appeared first on Data.gov.

Growing from 47 datasets in 2009 to nearly 450,000 datasets today, Data.gov reaches across 172 federal agencies to bring data to innovators, developers, analysts and citizens across the nation. The data shows up in smart phone apps, websites, and information that lets people buy smarter, use energy more efficiently, and find better health-care solutions each day.

Over the past year alone, Data.gov has not only added more datasets, it has added more opportunities for interaction with them, and more opportunities for collaborating and sharing information both nationwide and around the world. In fact, President Obama’s Open Government U.S. National Action Plan considers Data.gov an important tool to spur innovation in the United States and around the globe.

Data.gov has become a gathering spot for those with shared interests through its topic-based communities. Expanding these communities is a key priority of President Obama’s U.S. National Action Plan, which heralds six Data.gov communities: – Education, Health, Law, Energy, Safety, and Research.

These communities bring together experts from the public, academia, industry, and government to address the national challenges in energy, health, and law, and this year new communities launched on safety, education, manufacturing, oceans, ethics, developers, and business. From organizing challenges to inspire new innovations to supporting code-a-thons in cities, to building platforms for entrepreneurs to find new technologies and grow their businesses—Data.gov is putting federal data to work for Americans.
Join the celebration, discover new information, and find ways to get more from your money, your time, and your business. Visit Data.gov!

Data.gov is managed by the General Services Administration’s Office of Citizen Services and Innovative Technologies working with the U.S. Chief Information Officer and U.S. Chief Technology Officer.

The post Data.gov Celebrates Third Anniversary appeared first on Data.gov.

 Within months, Data.gov will see major enhancements around its data catalog, how the data is managed and the ease with which the data can be used. The most obvious change visitors will find is a single catalog that combines Data.gov and Geo.Data.gov. For the first time, nongeospatial datasets and geospatial datasets will be in one place. You’ll also find improved search capabilities, with a broader reach across the Executive Branch. Thanks to “Data.gov 2.0” code’s incorporation into the Open Government Platform (OGPL), Data.gov will be even more open and compatible with sites around the world as well.

 These new features are due to the software that will support Data.gov’s data management: CKAN. CKAN is an open-source data management system used around the world, including by the United Kingdom (U.K.) and by Europe’s brand new open-data site. Its developer, the U.K.-based Open Knowledge Foundation, is a nonprofit that works with people worldwide to promote increased transparency and engagement, in part through making CKAN available for open data sites.

 A launch of the new and improved Data.gov is planned for later this spring.

The post CKAN: the Horizon for Data.gov 2.0 appeared first on Data.gov.

By Sarah Gearen

From shopping for an airline ticket to choosing a college or purchasing a home, consumers face increasingly complex choices in today’s marketplace. Too often, the effort required to make informed financial decisions leaves many Americans in the dark – and often paying a price, missing a better product or surprised by hidden fees.

This data community is a centralized portal to federal government data and resources to empower consumers and spur innovation among entrepreneurs. In fact, several companies have already shared stories about how this data has enabled them to build apps and other tools that help consumers make more informed financial decisions.

Here are a few examples of just how entrepreneurs are leveraging the Administration’s open data efforts to build these tools:

Using the Bureau of Labor Statistics’ Occupational Employment Statistics, GetRaised offers consumers a tool that enables them to determine whether or not they are underpaid, and assists them in creating a raise request to discuss the possibility with their supervisors.  To date, GetRaised reports that nearly 70 percent of women who have submitted raise requests have succeeded in obtaining raises, at an average of $6,700.  

HelloWallet pulls data from the Federal Reserve System’s Survey of Consumer Finances and the Bureau of Labor Statistics’ Consumer Expenditures Survey to provide consumers with information about the financial habits of their peers.  The company’s aim is to encourage healthy financial behavior by presenting consumers with comparative mortgage, retirement and credit card data, and offering suggestions about where to save and reduce debt. 

Calcbench is a financial information and analytics platform that uses free eXtensible Business Reporting Language (XBRL) data from the Securities and Exchange Commission to deliver financial information about 8,500 companies listed on US-based stock exchanges.  By using XBRL data, Calcbench is able to offer everyday investors detailed information that has been traditionally been reserved for institutional investment firms to help them make smarter, more informed decisions on everything from business strategy to retirement investing.  

By combining data with technology, these companies are able to offer data analytics tools that are customized to address the goals and financial needs of individual consumers.  If your company or organization has a story to share about how you are using data to empower consumers, tweet us @USTreasury.  

Sarah Gearen is a Senior Policy Advisor in the Office of Consumer Policy at the U.S. Department of the Treasury.

The post Innovators Using Federal Data to Help Consumers Make Informed Decisions appeared first on Data.gov.

Posted by Dr. Catherine Woteki, USDA’s Chief Scientist and Under Secretary for Research, Education and Economics, on April 29, 2013 at 2:45 PM.

The opening day of the G-8 International Conference on Open Data for Agriculture was action-packed and inspiring. From the moment the doors opened at 7:30 am, the air was punctuated with the sound of languages from across the globe. Scientists, policy makers, and leaders from the non-profit and development community all shared a day of discovery and connection around the unlimited opportunity in open data for agriculture.

Secretary Vilsack kicked off the proceedings with a speech that focused the day. “Data is quickly becoming one of the most important commodities in agriculture,” he told the attendees, and encouraged the sharing of data to magnify its power.  He also compared the digital revolution fueled by open data to the industrial revolution, in that data sharing has the same potential to accelerate development of new tools that will bolster the productivity of farmers around the world.

As head of the U.S. delegation, he announced the launch of the federal government’s contribution to these efforts: the Food, Agriculture and Rural data community, which offers a catalog of over 300 data sets as well as numerous apps, tools and statistical products. The community can be found onwww.data.gov.

The community will provide a one-stop shop where the public can come to access all the available data relevant to agriculture and have come from federal research and investments. The community was launched in hopes that farmers, researchers, data analysts, businesses and governments will use it to locate, download and utilize food, agriculture and rural data to address serious social, economic and environmental concerns.

The Secretary was followed by Bill Gates, who addressed the conference via a short video presentation. Gates emphasized the importance of open data to entrepreneurship and innovation.  Other speakers included U.S. Chief Technology Officer Todd Park and Vice President of the World Bank Dr. Rachel Kyte.

Then the day took off with “lightning presentations” of apps already created using open data, and already working for farmers, scientists and policymakers.

Other announcements made by day’s end from our G-8 partners and others include a commitment from the African Orphan Crop consortium to share genetic sequences of neglected crops important to food security in Africa, and new data releases from the International Food Policy Research Institute, and the International Center for Tropical Agriculture with the Consultative Group on International Agriculture Research.

I hope you’ll join us tomorrow at the conference, which will be live-streamed at here and follow us on Twitter, #OpenAgData.

To read Secretary Vilsack’s speech click here.  To learn more about Open Data click here. Photos for the event can be found here.

The post Open Data for Agriculture Offers Lift-Off for Global Food Security appeared first on Data.gov.

But this is not enough. On May 9th Barack Obama ordered that all data created or collected by America’s federal government must be made available free to the public, unless this would violate privacy, confidentiality or security. “Open and machine-readable”, the president said, is “the new default for government information.”

The post Open Data: A new goldmine | The Economist appeared first on Data.gov.

Data.gov is also launching a new data catalog on an open source data management system called CKAN  The new catalog features a number of enhancements, such as an improved search that helps you find all the datasets for a particular location, better sorting and tagging of datasets, and more robust metadata. Data.gov now has one unified data catalog based on an open source standard that will make it easier to federate with other federal agency catalogs, as well as those of states, cities, and counties. 

Click here to see full blog post.

The post Data.gov Launches New Catalog and APIs appeared first on Data.gov.

A huge milestone was reached yesterday with the relaunch of the U.S. government data portal on a single, open source platform. A joint collaboration between a small UK team at the Open Knowledge Foundation and data.gov, this was an ambitious project to reduce the numerous previous catalogs and repositories into one central portal for serious re-use of government open data.

Catalog.data.gov brings together both geospatial as well as “raw” (tabular or text) data under a single roof in a consistent standardised beautiful interface that can be searched, faceted by fomat, publisher, community or keyword as well as filtered by location.

The post OpenGov Voices: Data.gov relaunches on open source platform CKAN | Sunlight Foundation Blog appeared first on Data.gov.

The post White House Issues Executive Order on Open Data | AIDS.gov Blog appeared first on Data.gov.

This Executive Order and the new policies that accompany it cover a lot of ground, building public reporting systems, adding new goals, creating new avenues for public participation, and laying out new principles for openness, much of which can be found in Sunlight’s extensive Open Data Policy Guidelines, and the work of our friends and allies.

The post Open Data Executive Order Shows Path Forward appeared first on Data.gov.

The post Tour Data.gov 2.0 appeared first on Data.gov.

Why? On May 9, 2013, President Obama issued an Executive Order (and OMB issued an accompanying policy memo) making “open” the new default for non-sensitive federal government data.

Before this shift, agency representatives would log on to a central database at Data.gov and enter information about the datasets they wanted to make public. All of these dataset entries are preserved and you can search them on Data.gov today – but no new datasets will appear on Data.gov until November 9, 2013.

Going forward, federal agencies need to maintain their own data inventory listings at agency.gov/data by November 9, 2013. Starting November 9, Data.gov will regularly crawl all of these inventory listings to power one central search engine at Data.gov, eliminating the manually-updated central database. Until this transition point, all crawled agency inventory listings will be searchable at Next.Data.gov. As agencies bring their new data inventory listings online and/or add new dataset listings, they will appear on Next.Data.gov (and not on Data.gov).

On November 9, 2013, Next.Data.gov will become the new Data.gov.

As we get closer to November 9, more and more agencies will have their data catalogs up, and we anticipate that every agency catalog will include all of the datasets originally listed on Data.gov as well as newly-listed data. In other words, the closer we get to November 9, the more robust the search results on Next.Data.gov will become.

Until November 9, we recommend searching both Data.gov and Next.Data.gov for the most comprehensive results.

The post Indexing Datasets on Next.Data.gov appeared first on Data.gov.

Last week, President Obama talked about the importance of opening government data to the American people, so entrepreneurs and business owners can use those troves of information to create jobs and solve problems that government can’t solve by itself. In remarks last Monday, he noted many examples of the Administration’s progress toward making that happen, saying:

“At Data.gov, you can search through and download more than 75,000 data sets – data on everything from what different hospitals charge for different procedures, to credit card complaints, to weather and climate measurements.”

Today, we’re excited to share a sneak preview of a new design for Data.gov, called Next.Data.gov. The upgrade builds on the President’s May 2013 Open Data Executive Order that aims to fuse open-data practices into the Federal Government’s DNA. Next.Data.gov is far from complete (think of it as a very early beta), but we couldn’t wait to share our design approach and the technical details behind it – knowing that we need your help to make it even better.

The post First Look at Next.Data.gov | White House Blog appeared first on Data.gov.

The post Agencies get how-to on open data compliance | FCW appeared first on Data.gov.

The administration’s document, posted on the GitHub website, focuses on near-term efforts agencies must take to meet the five initial requirements of OMB Memorandum M-13-13 (.pdf), which are due November 1, 2013.

The post White House issues guide … Open Data Policy | Fierce Government IT appeared first on Data.gov.

The Climate Corporation offers insurance, software, and services to help farmers plan, manage, and protect their crops by using a number of open federal government data sets, including weather history and forecasts from the National Weather Service, terrain maps and soil type from the U.S. Geological Survey, soil quality from the Natural Resources Conservation Service of the U.S. Department of Agriculture (USDA), crop yields from USDA, and weather and earth observation data from NASA.

Mercaris uses open data from USDA to deliver market data and online trading tools that can assist in pricing decisions for the organic and non-GMO food industry.

FarmLogs offers analytical tools to help farmers improve on-season and off-season decision making by using several open data sets, including weather and GPS data from NOAA and pricing and news from the Chicago Mercantile Exchange.

The post Apps for Farmers appeared first on Data.gov.

The post MarineCadastre.gov appeared first on Data.gov.

The post National Agriculture Statistics Service QuickStats API appeared first on Data.gov.

The post Economic Research Service ARMS API appeared first on Data.gov.

The post Agricultural Research Service Germplasm Resources Information Network appeared first on Data.gov.

The post Food and Nutrition Service appeared first on Data.gov.

The post Agricultural Productivity in the United States appeared first on Data.gov.

The post Farmers Market API appeared first on Data.gov.

The post APIs from the Energy Information Administration appeared first on Data.gov.

The post Agricultural Marketing Service appeared first on Data.gov.

– See original blog by Joyce Hunter, Deputy Chief Information Officer for Policy and Planning

The post From Data to Decisions: Using Data to Improve Public Access and Knowledge appeared first on Data.gov.

The post Dive Deeper into USDA Data with New APIs appeared first on Data.gov.

As this is posted today, the Data.gov team has re-located to our new “greener” General Services Administration (GSA) building in One Constitution Square in Northeast Washington, D.C.   These new surroundings, optimized for open collaboration, are filled with new opportunities and possibilities.  This new work environment epitomizes the key characteristics of the Data.gov team – nimble, collaborative, adaptive, and forward-leaning (to new horizons)!  Our physical address may have changed but our digital, virtual addresses (e-mail, phone, PDA, and web) in the digital domain remain the same as our transition was seamless and flawless.

The International Open Government Data Conference (IOGDC) was a Rousing Success!

By all accounts and feedback, the International Open Government Data Conference (IOGDC) held in November in Washington DC was a rousing success!  It attracted nearly 400 people from around the world to discuss, explore, and celebrate the great strides taken in opening up government data.  From Australia to the United Kingdom, from Poland to Vietnam, experts shared the lessons they had learned in creating open data movements, developing apps, and analyzing government data.  Many thanks to everyone who attended and made those three days such a rich experience for all of us.

Data.gov sponsored the conference to bring together so many of the people who have been meeting independently in this area.  Recently, developers have been hosting and attending “hack-a-thons” and “mash-a-thons.”  Government executives have been meeting to formulate policy decisions.  Security experts have been investigating the issues related to the release of data.  But … these have happened independent of each other.  The IOGDC brought many of these people together to share the conversation and shape the vision for the future of open government data.

We converged on the U.S. Department of Commerce auditorium, we shared knowledge formally and informally—believe it or not, open government data people are incredibly passionate and committed about their work—and we launched a new community among those committed to using and making available government data worldwide.  Check out the postings in the community site or review the “blow-by-blow” events from the >1,700 tweets on #IOGDC!

Dan Melton of Code for America captured the passion of conference participants!

I appreciated the participation and perspectives of visionaries U.S. Government Chief Information Officer Mr. Vivek Kundra and World Wide Web inventor Sir Tim Berners-Lee at the IOGDC

Open Government Data Perspectives and Progress around the Globe were Illuminating

Dave McClure from GSA, myself, Richard Best from New Zealand, Trevor Smallwood from Australia, and Stephen Walker from Canada sharing lessons learned about open government data implementations.

Nearly 400 participants came from 5 continents, 13 nations, and 31 federal agencies.  It was evident that a lot of progress has been made in the U.S. and abroad since the launch of the Open Government Initiative and Open Government Directive (OGD) and Data.gov in 2009.  The paradigm shift from “protect everything and share what you must” to “share everything and protect what you must” (but still comply with National Security, Privacy and Information Quality Regulations) based on the President’s vision and the Attorney General guidelines were important to the open data movement and phenomenon.  Hearing about the great work by each of the government organizations in the U.S. and abroad was the highlight of the conference.   The progress was palpable and measurable.

You should check out the amazing progress made by the participant countries and how they are addressing challenges and opportunities with pragmatic solutions:

1. Protecting Security and Safeguarding Privacy in the Age of Open Government:  Office of the Director of National Intelligence (ODNI), GSA, Office of Management and Budget  (OMB), and Australia*
2. Data Sharing to Solve Mission Problems—Are We Doing Enough?: U.S. Environmental Protection Agency (EPA), Johns Hopkins University, and GSA
3. The Semantic Web—What is it? Is it Ready for Prime-Time? And How Can I Use It?: Rensselaer Polytechnic Institute (RPI), U.S. Department of Health and Human Services (HHS), GSA and United Kingdom*
4. Putting Data to Work—Creating Value-Added Functionality and Applications Through Mash-ups: RPI, U.S. National Renewable Energy Laboratory, U.S. National Institutes of Health, and GSA
5. Harnessing Data to Inspire and Inform: Code for America and New York Times
6. Building a Blueprint for Collaboration—A Healthy Approach to Open Data: ODNI, HHS, and GSA
7. On the Road Again—Data Available Anytime, Anywhere Through Mobile Apps: University of Illinois Urbana-Champaign, GSA, and RPI
8. Case Study on Data.gov: Harvard Business School, Harvard University
9. What is Value Proposition of Open Government Data? How Do We Measure Success?:U.S. Department of Interior (DOI), University at Albany, O’Reilly Media, and United Nations*
10. Technologies for Transparency—Dynamic Open Data Publishing with Open APIs: Clark & Parsia LLC, Code for America, New York Times, HHS, and U.S. National Aeronautics and Space Agency (NASA)
11. Enabling Geospatial Killer Apps—Interfaces, Visualizations and APIs—Imaging the World: GSA, U.S. Geological Survey (DOI), and EPA
12. How To Create A Vibrant Open Data Community: U.S. Department of Education, U.S. Department of State, Sunlight Foundation, and GSA
13. Open Government Sites—Where We Have Been and Where We Are Going: New York State Senate, GSA, and Brazil*
14. Working Together In Communities of Practice with Metadata Standards: Federal Reserve System, Bureau of Labor Statistics, and OMB
15. International Open Government Data Leaders—Top 10 Issues and Lessons Learned: GSA, United Kingdom, New Zealand, Australia, and Canada*

Senior U.S. leaders from the White House delivered four keynotes with the following themes:

1. Data.gov importance to open government and the road ahead
2. Government-wide Open Government Directive and open data progress
3. Closing the technology gap with platforms and participation to deliver ecosystems focused on results
4. The global impact of open data

It is noteworthy that international organizations participated in five of 15 session areas noted above* with equal contributions and important perspectives.  Additionally, International organizations led three other sessions with exclusive international perspectives.  You should check out the huge strides made by the international organizations, which are embracing Open Government Data and addressing challenges:

• New Zealand and Australia addressing “licensing and ownership of government data”
• New Zealand focused on social media and open data policies
• The U.K. in creating semantically rich datasets on Data.gov.uk
• Brazil for e-democracy and participative government activities
• Australia in making government both secure and transparent
• The United Nations (UN) programs for e-governance that reaches to developed and developing countries
• Canada in opening up both national and provincial data
• The World Bank was very innovative in open development with open data, open knowledge and open solutions.  Their “Apps for Development” contest was noteworthy

It is obvious that we are forming a powerful Open Data community across the globe.  People have already contacted me about the next IOGDC event based on the incredible amount of positive energy, knowledge sharing, goodwill, and possibility for greater good that was experienced at the conference.

Key Epiphany:  At the risk of waxing philosophical, I have observed that we tend to learn from each other continuously.  No one has the monopoly on the best ideas.  We keep building on each other’s ideas, innovations and success that can become a force multiplier for a greater good!  As Spock said, “The needs of the many outweigh the needs of the few or the one.”  But you knew that.…

Practice what you preach!

We’ve been practicing what we have been preaching:

1. You can find all the speakers listed and presentations here for the three days of amazing discussions!  We loaded all the presentations immediately on the Data.gov website and we were lauded by many for our openness, transparency, and quick response
2. We participated and coordinated with the Open Government Data Camp: 2010 that took place in London, U.K. that week, speakers traveled from one conference to the other, we cross-posted twitter hash tags, we held discussions in both of the community spaces, and we have created an ongoing series of conversations at multiple levels (policy, technical, developers, and citizens) to grow a truly international view and shared space of open government data.

Finally, we tracked the efforts of the International Open Data “Hack-a-thon” day on December 4, 2010, which attracted thousands of people in over 73 cities on five continents.  Application developers from around the world looked at data available from many data.gov sites to create better ways to get information to citizens to make decisions.

Harvard Business Case Study Provides Key Insights on Data.gov

I mentioned last time that Harvard Business School had published a case study about Data.gov.  As Data.gov’s senior executive, I was especially proud and pleased to present Harvard Business School (HBS) Assistant Professor Karim Lakhani at the IOGDC, who discussed the recent HBS case study on Data.gov, the full text of which is available on the Data.gov site. Harvard studied Data.gov to better understand Federal CIO Vivek Kundra’s groundbreaking ideas behind it:

• Move government data to the web
• Make data available in raw, machine-readable format
• Encourage people to use the data
• Harness the wisdom of crowds beyond government to drive innovation

What was the result? BETTER GOVERNMENT!

The HBS case study pulls together the best history—word that seems slightly pretentious since we have barely celebrated our first anniversary—of Data.gov from the major players. We have accomplished much in the past year: hosting more than 300,000 datasets, launching apps competitions based on the data, and encouraging use of the data by researchers, developers and the greater public–“putting the data to work” is how we like to describe it.  However, the HBS assessment of the way forward is what excites me most.

The HBS case study brings in voices from across the open government data environment, including Ellen Miller from the Sunlight Foundation, Tim O’Reilly of O’Reilly Media, Dave Campbell of Microsoft, along with those of the U.S. Federal government, to discuss Data.gov’s challenges and opportunities. We are in new, uncharted territory. How do we make the most data available while maintaining individual privacy and security? How do we find and assemble a community of experts to create best practices in using U.S. government data? We feel that the recent conference was a big step in this direction.

It’s time to take the open data movement to the next level—we need to create the international communities, topics, and focus areas that need to be addressed as a global community.  I encourage you to read this case study and contact me with your ideas—let me know what you’re doing so we can move forward together.

Stay involved and tuned—new communities of data are coming …

Join in the discussions in the open data or semantic communities and stay tuned as new communities in health, economics, climate, and law come to life in the next quarter of fiscal year 2011….

Many on our team will take some much-needed leave this month after working assiduously and continuously for a long while – we shall all re-emerge re-charged, renewed and rejuvenated in 2011!  I wish all of you a safe, joyous, and prosperous Holiday Season!   Let good data set you free …

The post “Think Globally – Act Locally”- A New Chapter for Data.gov… appeared first on Data.gov.

The post Global Agriculture Data appeared first on Data.gov.

The post Federal Data Resources appeared first on Data.gov.

The post AIS Data for Vessel Traffic Patterns appeared first on Data.gov.

The post Energy Data Initiative appeared first on Data.gov.

The post Global Open Data Initiative for Agriculture and Nutrition… appeared first on Data.gov.

The post USDA Blog appeared first on Data.gov.

The post Dr. Woteki on Scientific Integrity appeared first on Data.gov.

As of May 2011, the available editions of eGRID contain data for 2007, 2005, and 2004 (all in eGRID2010) and for 1996 through 2000 (in the archived edition).  Data for 2007 were configured to reflect the electric power industry’s current structure as of December 31, 2010, including plant ownership and operators, parent company affiliations, company mergers, and grid configurations. TranSystems|E.H. Pechan, formerly E.H. Pechan & Associates, Inc., has developed all seven editions of eGRID for EPA since 1997.

Data Summary

eGRID data include emissions, emission rates, electricity generation, resource mix, and heat input.  eGRID data also include plant identification, location, and structural information. The following emissions information is included in eGRID:

• carbon dioxide (CO₂)
• nitrogen oxides (NO_x)
• sulfur dioxide (SO₂)
• methane (CH₄)
• nitrous oxide (N₂O)
• carbon dioxide equivalent (CO₂e)
• mercury (Hg)

CO₂, CH₄, and N₂O are greenhouse gases (GHG) that contribute to climate change. NO_x and SO₂ contribute to unhealthy air quality and acid rain in many parts of the country. Hg is a toxic pollutant that can adversely affect exposed nervous systems. 

eGRID’s resource mix information includes the following resources: 

• coal
• oil
• gas
• other fossil
• nuclear
• hydroelectric (water)
• biomass (including biogas, landfill gas and digester gas)
• wind
• solar
• geothermal

The primary data sources used for eGRID include data reported by electric generators to EPA’s Clean Air Markets Division (pursuant to 40 CFR Part 75) and to the U.S. Energy Information Administration (EIA).

Uses of eGRID

eGRID data are used for carbon footprinting; emission reduction calculations; calculating indirect greenhouse gas emissions for The Climate Registry, the California Climate Action Registry, California’s Mandatory GHG emissions reporting program (AB 32) , and other GHG protocols.   EPA tools and programs such as Power Profiler (which has been updated with 2007 data from eGRID), Portfolio Manager, the Green Power Equivalency Calculator, the Greenhouse Gas Equivalencies Calculator, the Personal Greenhouse Gas Emissions Calculator, and the WasteWise Office Carbon Footprint Tool use eGRID.  Electricity labeling/environmental disclosure, Renewable Portfolio Standards (RPS) and Renewable Energy Credits (RECs) attributes are often supported by eGRID data.  State and local governments also rely on eGRID data for emissions inventories and for policy decisions.  eGRID is used extensively for electric grid related research by nongovernmental organizations.

 In 2010, Executive Order 13514 was issued, requiring Federal agencies to “measure, report, and reduce their greenhouse gas emissions from direct and indirect activities.” The Federal GHG Accounting and Reporting Guidance accompanied this order and recommended using eGRID non-baseload emission rates to estimate the Scope 2 (indirect) emission reductions from renewable energy.

The company SAP has taken eGRID data and made an online visualization of eGRID subregion, NERC region, and state data.  See:  http://experience.sap.com/experience/html/Pages/egrid/dashboard.html .  This application is under review.

Accessing eGRID Data

In addition to Data.gov, eGRID is also available on EPA’s website at epa.gov/egrid.  Year 2007 data in eGRID2010 are presented in three Excel workbooks. The first workbook contains data at the boiler, generator, and plant levels. The second workbook presents aggregated data by state, electric generating company, parent company, power control area, eGRID subregion, NERC region, and the U.S.  The third workbook contains state import-export data for 2004 and 2005, as well as U.S. generation and consumption data.  The eGRID2010 Technical Support Document, File Structure, Summary Tables (with regional representational maps), and GHG Annual Output Emission Rates for CO₂, CH₄, and N₂O (for GHG inventory and registry work) are included in Adobe PDF files. These files, along with 2005 and 2004 data, are contained in a ZIP file that can be downloaded from EPA’s eGRID website. 

Future Plans

The 2009 eGRID data are presently being developed with a planned release in early 2012.

Do you have a good idea for an app using eGRID data?  See EPA’s Apps for the Environment Challenge:
http://www.epa.gov/appsfortheenvironment/ .

Author information

Art Diem is an environmental engineer who manages eGRID for EPA.  Dr. Susy S. Rothschild, a senior scientist at TranSystems|E.H. Pechan, formerly E.H. Pechan & Associates, Inc., is the primary contracting staff for eGRID since its inception.  For questions or comments about eGRID, please use the eGRID feedback page.

The post Creating the Emissions and Generation Resource Integrated Database (eGRID) appeared first on Data.gov.

The post Rooftop Solar Challenge appeared first on Data.gov.

The post Code for Livability appeared first on Data.gov.

The post America’s Home Energy Education Challenge appeared first on Data.gov.

The post Challenges appeared first on Data.gov.