NOAA Responds to Peer-Reviewed Critique of the Federal "Billion Dollar Disaster" Metric

Please click the ♡ button if you appreciate my work. This helps boost post visibility.

This is the 2023 “billion dollar disasters” visual summary published by the National Oceanic and Atmospheric Administration in January and here’s the home page for this longstanding and much-cited metric tracking impacts from climate-related disasters:

Roger Pielke Jr.

, whom I interviewed in January about disaster trends in a human-changed climate (see below), has had his paper critiquing this federal “billion dollar disaster” metric published in the peer-reviewed Nature journal NPJ Natural Hazards. Here’s his post offering key takeaways and links to the paper etc.:

Scientific integrity and U.S. “Billion Dollar Disasters”

I sent the new paper with a request for comment and for an eventual Sustain What conversation to the National Oceanic and Atmospheric Administration press office and Adam B. Smith, the NOAA scientist who’s the main architect of the analysis.

Here’s Smith’s reply, copying the press officers, which includes ample signals that the processes that sustain the quality of government information - including input from independent experts like Pielke, will lead to updates:

Below is a NOAA response.  We are also considering a response article in the same Nature journal, to clarify aspects of the product.  

Also, I would be happy to join your webcast around the time our response article is published, to help others better understand our data and methods.

NOAA is aware of recent media reports related to the accuracy of the BDD [Billion Dollar Disasters] product. The Department and NOAA take information quality and scientific integrity considerations very seriously and have established a robust process for addressing Information Quality Act requests for correction and scientific integrity allegations (refer to NOAA’s Scientific Integrity Commons for details on the process, https://sciencecouncil.noaa.gov/scientific-integrity-commons/). NOAA is working on a couple of these currently, and will be issuing findings once the formal process is completed. NOAA will then address any findings as part of our constant review and improvement of our products and services.

The BDD product is based on two decades of research and close collaboration with public and private sector partners, and we engage with the user community regularly on how to improve it. For example, NOAA held a special session at the American Meteorological Society Summer Meeting in August 2023 to seek input on filling gaps and making improvements to the product.

The feedback included adding information for the many small and medium-sized event costs to the larger, costlier billion-dollar disaster event data that NOAA historically provides.

Participants also discussed whether NOAA should explore how the growth in our country’s exposure and vulnerability to weather and climate extremes is changing regionally over time. NOAA is working to include such data into our analysis. NOAA welcomes this feedback as it will continue to improve these products and our ultimate service to the Nation.

Best regards,

Adam Smith

Applied Climatologist (LinkedIn)

Climate Science and Services Division

National Centers for Environmental Information (NCEI)

Added after posting: I noted the response was tailored to earlier press coverage of the pre-review version of Pielke’s paper and Smith replied:

Yes, that statement was largely reflecting the non-reviewed version that was floated online earlier this year.

Also, it seems that the final published version is quite similar to the earlier version.  

Indeed, there are several misunderstandings from the author's point of view that we will clear.

The link under Smith’s name in his email signature goes to this Weather Channel discussion of the billion-dollar disaster metric, reposted on a NOAA Youtube channel. It’s worth watching:

Here’s a key section from Pielke’s post:

I enjoyed working in a NOAA cooperative institute at the University of Colorado Boulder for 16 years. I have many friends, colleagues, and collaborators in the agency. NOAA does excellent work — across both research and operations — and every minute of every day contributes positively to making the nation safer and furthering its economy. That makes the agency’s failures on the “billion dollar disasters” so remarkable and troubling.

Here is how I end the paper:

“NOAA is a crucially important agency that sits at the intersection of science, policy and politics. It has a long and distinguished history of providing weather, climate, water, ocean and other data to the nation. These data have saved countless lives, supported the economy and enabled significant scientific research. The agency is far too important to allow the shortfalls in scientific integrity documented in this paper to persist. Fortunately, science and policy are both self-correcting.”

I’ll be watching closely to see how NOAA responds to this new paper, how (and if) the media continues in its promotion of the “billion dollar disaster” tabulation in light of this new peer-reviewed analysis, and what sorts of self-correction might take place among policymakers who have used the dataset to justify specific policies.³

You can read the paper in full here and access and an enhanced PDF here — all open access.

Here’s a relevant snippet from my January chat with Pielke and the January post with the rest: