Where does propagation data come from?

Foundations of Amateur Radio One of the many questions that new amateurs ask is, "When should I get on-air, and on what band?" The often-heard reply is just to get on-air and make some noise. As time goes by, the importance of this seems to fade in favour of using HF prediction tools. Some amateurs never venture beyond that point, relying almost exclusively on technology to determine if they should turn on their radio or not. If you search the internet for "current HF conditions", you'll end up with dozens of sites boldly claiming to provide precisely that information, some even using the label "Real-Time". You'll find instructions from countless self-proclaimed "experts" on how to read propagation conditions from their favourite site. There's even widgets that you can install on your website displaying propagation data per amateur band with helpful labels like "Band Closed" or showing conditions as "Poor", "Fair" or "Good". Some of these widgets even include an embedded time-stamp to prove just how "current" the information is. If that's how you decide to activate your amateur station, like I once did, I have some questions. Where is this information coming from, is it accurate, and when was it last updated? To give you an idea of just how complex this question is, consider visiting two popular websites, solarham.net from Canada and spaceweatherlive.com from Belgium. On their home-pages, you'll find all manner of numbers, charts, photos, events, notifications, alerts, and warnings, each related in some way to HF propagation and the condition of the Sun. Sounds great, excellent resources, job done. Well, no. Let's start simple. Location. Leaving aside where the site's owner is or where the servers are, both potential sources of confusion, consider where you are and where the remote station is that you're trying to contact. Now compare that with the propagation data location. Do you know where the measurements came from and if they're relevant to you? What about data currency? For example, if you can see the Sun, you can count the number of sunspots since that data comes from physically looking at the Sun. Mind you, can someone count the number of sunspots at night? It's not a trick question. The Sun isn't overhead for everyone all the time, and the data from any particular observer will be out of date at night. When was the count updated? Is it still actually current, let alone real-time? Obviously, not everyone uses the same data source either. In case you're wondering, why are we counting by eye in the space age? It turns out that, since Galileo more than 400 years ago, it's the most long-term, reliable way to keep data consistent between observers and instruments, both of which often last only one or a few solar cycles, and it's also cheap! What about equipment changes and failures in data gathering? Geomagnetic activity isn't global; it's measured using a device called a flux-gate magnetometer. Measurements from specific instruments scattered around the globe are combined into the planetary, or Kp index. You'll discover that locations used change over time, and when instruments are down, the numbers are estimated, but you won't see that unless you actually find and explore the source data. It's not just solarham.net and spaceweatherlive.com; it's pretty much every single site that shows any form of HF propagation or space weather information. Even sites based in a specific country, like the Australian Space Weather Service, have many instruments scattered around Australia. If you happen to be near an actual instrument, where "near" is anything less than 500 km away, how do you know if that instrument was actually online when a measurement was made? Even if the instrument near you is working, is the data relevant to the receiving station on the other side of the planet? If you look closely at the sites giving out current HF conditions, you'll discover that most of these don't even tell you where the data comes from, let alone if any of it was estimated to come up with their current reported values or recommendations. If you start searching for historical information, this problem gets bigger. You'll find many sites that claim to have data, but are invariably underfunded, are rife with broken links, out-of-date servers, and moved, deleted, and abandoned pages. If you unearth a dataset, you'll discover that everyone uses a different standard to record their measurements. How do you even know if combined measurements are coming from the right column? Think I'm kidding? There are documents with warnings about different formats, calculations, and dates on which these changed. Aggregating this data is challenging, at best. So, is there a better way? Yup. You're not going to like it. "Get on-air and make noise!" I can hear you groaning from here. It's not all bad. You can run your own beacon to see the conditions at your location. It's what started me down the path of installing a WSPR, or Weak Signal Propagation Reporter, beacon and leaving it running 24/7. Currently, I'm focused on very weak, 10 mW signals. So far, it's been reported 3,685 km away. If you visit the VOACAP or Voice of America Coverage Analysis Program website, you'll find a visualisation of how FT8 propagation worked between ITU zones between 2017 and 2019. It's not current, but it's an excellent way to see how propagation data can be derived from actual contacts. What we really need are more beacon transmitters and online receivers. I'm Onno VK6FLAB.