PODCAST · technology
Hacker Public Radio
by Hacker Public Radio
Hacker Public Radio is an podcast that releases shows every weekday Monday through Friday. Our shows are produced by the community (you) and can be on any topic that are of interest to hackers and hobbyists.
-
10
HPR4728: Programmable Logic Controls - Episode 3
This show has been flagged as Clean by the host. -------------------- 01 Introduction This is the third episode in an 8 part series. 02 In the previous episode we covered * The early history of computers in industrial control * The early history of PLCs, including how they got their name * Who the major brands are * What they look like physically * A basic description of the abstract machine architecture * A very brief look at what a PLC program is like * The scan concept * The main PLC programming languages * The minor PLC programming languages * The relative popularity of each of the programming languages 03 In this episode we will begin by taking a look at one of the early PLCs from the era when they first began seeing widespread use. I will not even attempt to try to be comprehensive, I will just give a broad overview in the limited time I have available here. I have started with an older model because its simplicity and limited features allow for an easier introduction to the topic. -------------------- 04 Allen Bradley PLC/2 The Allen Bradley PLC/2 was introduced to the market in 1977. It was not their first foray into this field, but it was their first really successful one. I am not aware of a comprehensive history of their early product line, but one of their early major selling models was the PLC2/30. This is also known as their 1771 series. -------------------- 05 Physical Layout The CPU module was a large metal box which sat beside the I/O rack and connected to it with a cable. The I/O rack was a box with an open front and an series of slots into which tall narrow boxes containing the I/O could be slotted. 06 A backplane ran across the back, connecting the I/O to the CPU. Multiple racks could be connected together by cables. For the PLC2/30, you could have a maximum of 896 digital I/O points. 07 Racks were 315mm high, and between 247mm to 610mm wide, with racks coming in 4, 8, 12, and 16 slot varieties. The CPU was the same height as the rack and roughly square in outline. 08 As micro electronics advanced, the CPU module was able to be shrunk down in size such that it would fit into a slot in the rack, which became the norm for PLCs. -------------------- 09 The Electronics Inside Early model PLC/2s used some sort of 8 bit processor, some sources say an Intel 8080. 10 They also used four AMD 2900 bit slice processors as a logic co-processor. If you are not familiar with bit slice processors, these are chips which are like a 4 bit vertical slice through a processor, and can be joined together with logic chips to form a complete CPU. These are what were used to construct minicomputers. 11 They were used as coprocessors in early PLCs because microprocessors on their own were simply too slow to handle running the user program rapidly enough to allow a useful size program. One or more microprocessors were used as well in order to coordinate the overall operation and system management. 12 As microprocessors grew faster and more powerful, the need for logic coprocessors declined and they were eventually dropped. Early versions used magnetic core memory. Later ones switched to some sort of solid state RAM, probably static RAM of some sort. 13 Details of what sort of processors are inside any PLC are actually very hard to come by as manufacturers don't generally talk about that sort of thing. They wish the user to see it as just a black box. -------------------- 14 The Data Table From a programmer's perspective, the most important thing to understand first is the data table. The data table is the PLC's data memory. 15 For a PLC2, this is an array of 16 bit words. Each word consists of two 8 bit bytes. Addressing for both bits and words is in octal. 16 For those not familiar with them, octal numbers follow a counting system that goes from 0 to 7. The next number after 7 is 10. Counting then proceeds from 11, 12, 13, etc. going to 17, 20, 21, etc. Each octal digit takes exactly 3 bits. 17 Individual bits in PLC2 notation may be addressed by specifying the word followed by a slash, and then the bit. For example, 030/12 is the 12 bit in word 030. Remember that this is octal, so 12 is not the 12th bit if you are counting in decimal. 18 Memory Organization On the PLC2/30, the data table has the following organization. 19 Word addresses 000 to 007 are Processor Work Area number 1. This is not accessible by the user. 20 Word addresses 010 up to, but not including, 100 are the Output Image Table. This is a memory mapped image of the I/O outputs. 010 to 077 is for Rack 1. 020 to 027 is for Rack 2. This pattern continues up to Rack 7, which is 070 to 077. 21 Word addresses 100 to 107 are Processor Work Area number 2. This is also not accessible by the user. 22 Word addresses 110 up to, but not including, 200 are the Input Image Table. This is a memory mapped image of the I/O inputs. This is laid out in the same way as the output image table, and goes from 110 to 177. 110 to 117 is for Rack 1. 120 to 127 is for Rack 2. This pattern continues up to Rack 7, which is 170 to 177. 23 You will notice that the output image table and input image table appear to address the same rack slots. They do in fact do this. As to which address a specific slot in a specific rack maps to depends on whether there is an input card or an output card in that slot. 24 Addresses 200 to 277 are for Timer/Counter accumulated values. An accumulated value is the current time or count. 25 Addresses 300 to 377 are for Timer/Counter preset values. A preset value is the target time or count which when reached causes the timer or counter to indicate that it has reached the desired value. 26 The memory above 400 can be configured to split it into a data storage area and user program area. The data storage area is where you would store data that your program needs to use which is not part of the I/O or times and counters. You need to strike the correct balance between user data and program size. 27 There are various things that can be changed and configured with respect to the above, but I am not going to cover that in any depth as this is not a tutorial on the PLC2. = You should however have a pretty good idea of the memory of an early model PLC. The things to understand are that I/O are mapped to memory addresses, and different memory ranges are used for different purposes. 28 All memory management was manual. It was up to the user to keep track of which memory addresses were to be used for what purposes. Allen Bradley helpfully provided paper forms which you could photocopy which you could use to plan out and document what each address would be used for. Part of the programmer's job was to make efficient and logical use of memory, while also leaving space for future changes. -------------------- 29 The User Program The user program is made up of instructions. Each instruction typically takes one word of memory. However, complex instructions can take up to 8 words of memory. 30 There is a main program. You can think of this as like the "main" function in C. If you don't find that analogy helpful, then just think of it as this is where your program starts. The main program continues with one rung after another until it reaches the END statement. 31 There is also a subroutine area. The main program calls a subroutine by using a Jump to Subroutine, or JSR instruction. 32 The T3 Programming Terminal When the PLC2 came out, things like laptops were still far in the future. Even the first Compaq Portable suitcase style PC was still some years away. Indeed, the first PLC/2 came out not long after the first Altair PC kit. 33 Programming initially therefore was done using a special programming terminal known as a T3. The T3 was a suitcase size box with a small CRT in the end of it, and a keyboard attached to it below the screen. If this sounds like the early portable PCs, keep in mind that this in fact predated them by a number of years. 34 The keyboard was not a typewriter or QWERTY style. It had a membrane keypad with graphical symbols. Recall the previous episode on control diagrams using relays and how these were documented using schematic symbols on drawings. The T3 terminal keypad had symbols corresponding to a simplified and modified version of the symbols used in those schematic drawings. 35 When a programmer pressed one of those symbols on the keypad, the corresponding symbol appeared on the screen in the current cursor location. The programmer could then enter the data address that the symbol was to correspond to using the numeric part of the keypad. 36 An engineer, technician, or electrician could therefore create and enter programs using a fully graphical environment that used his existing knowledge of control using hardware components. They did not see this as programming a computer, they saw it as something else entirely which was unrelated to computers. 37 Programming took place with the T3 connected to the PLC while the PLC was in program mode. The instructions were entered directly from the terminal into the PLC memory. Programs could be saved or restored to cassette tape. The program would run when the PLC was set to run mode. 38 What is more, the T3 terminal allowed on line debugging of programs. When the T3 was connected in run mode, the ladder diagram would update live. When a logic condition was true, it would be highlighted. When the logic condition was false, it would be shown in an non highlighted state. 39 By observing a specific rung, you could see if the expected series of graphical instructions were all highlighted in a pattern which would result in the final output of the rung to be true or false as expected. This was an invaluable troubleshooting aid in finding both software bugs and hardware faults. Programming terminals became an indispensable part of an industrial maintenance electrician's essential equipment. 40 Eventually as portable PCs and later laptops became available, the T3 became obsolete and these functions were transferred to software that ran on standard PCs. However, they still work essentially the same way. Additional capabilities were added, such as the ability to write programs while not attached to the PLC. Descriptive labels and comments could be added to memory addresses to document them and aid in reading the program, Cross referencing functions would make it easier to find which addresses were used for what, etc. -------------------- 41 Instructions I will now turn to listing the sorts of instructions which were available to use. I won't try to explain everything, but it should at least give you a vague idea of how these were programmed. I will use Allen Bradley's names for these, which I should emphasize do not correspond to what other manufacturers call them. 42 Relay Type Instructions These are boolean instructions and consist of the following. 43 Input instructions These examine bits in memory and yield a logic state depending on the value of the bit they address. Input instructions start at the left of the rung and work their way right as more are added. Examine on - This is true when the addressed bit is set to 1. Examine off - This is true when the addressed bit is set to 0. 44 Output Instructions These turn bits on or off depending on the sum of the boolean logic conditions leading up to them. Output instructions are on the right of the rung. There must always be at least one output instruction. Output Energize -When the logic conditions leading up to it are true, it sets a bit to 1. When the logic conditions leading up to it are false, it sets a bit to 0. 45 Output Latch - When true, the bit is turned on. When false, it does nothing. Output Unlatch - When true, the bit is turned off. When false, it does nothing. 46 Branch Instructions These do nothing themselves, but they make wiring connections on the diagram. These are important to allow the instructions listed above to be placed in parallel in the same rung so that they can do things like specify OR conditions. You can think of them as essentially turning an examine on or examine off into an OR or OR NOT instruction. -------------------- 47 Timer and Counter Instructions Timers and counters are as the name implies, instructions which do timing and counting operations. They have separate preset and accumulated values. They are a type of output instruction in that they reside on the right side of the ladder rung and their operation depends on the state of the rung logic to their left. 48 Timers and counters work in Binary Coded Decimal, or BCD values. BCD values each take 4 bits of memory. However, unlike hexadecimal, only values from 0 to 9 are valid. What would be values from A to F in hexadecimal are not valid values in BCD. 49 The preset and accumulated values are stored in 16 bit words. The first 12 bits are used to store up to 3 BCD digits. The remaining uppermost 4 bits are used to store instruction status bits such as whether the timer or counter is done and various other features. 50 Timers The timer time base corresponds to the timer resolution. The time bases are 1.0 second, 0.1 seconds, and 0.01 seconds. So a timer with a 1 second time base can time from 0 to 999 seconds to one second of resolution. A timer with a 0.1 second time base can time from 0 to 99.9 seconds to 0.1 seconds of resolution. 51 There are three types of timer, Timer On Delay (or TON), Timer Off Delay (or TOF), Retentive Timer (or RTO). 52 On delay timers time while the rung condition is true and reset when it goes false. Off delay timers time after the rung condition goes false and reset when it goes true. Retentive timers are like on delay timers but retain their accumulated value until reset by the Retentive Timer Reset (or RTR) instruction. Generally, you start a timer and then monitor its "done" bit somewhere else in your logic. 53 Counters Counters count events. These come in the following varieties. 54 Up-counter (or CTU). Down-counter (or CTD) Counter Reset (or CTR) Scan Counter (or SCT) 55 Up counters count upwards from zero, and down counters count downwards from the preset. The scan counter counts the number of program scans. This is not something you would typically find a use for, although you can use it to for example calculate the scan time. -------------------- 56 Data Manipulation Instructions Data manipulation instructions are used to read, write, and compare byte and word data. Operations take place when the rung goes true. 57 GET reads a 16 bit word from a memory location. PUT writes a 16 bit word to a memory location. LES resolves to true if the value read by the GET instruction is less than a specified value. EQU resolves to true if the value read by the GET instruction is equal to a specified value. 58 The LES and EQU instructions can be combined in various ways with each other and with GET to give all the possible comparison permutations such as greater than, less than or equal to, etc. 59 GET BYTE reads a 8 bit byte from a memory location. LIMIT TEST resolves to true if a byte value is between the limits stored in the upper and lower bytes of a word. LIMIT TEST is used in combination with GET BYTE. -------------------- 60 Arithmetic Instructions Arithmetic instructions also operate on BCD word values. Operations take place when the rung goes true. The uppermost byte is used to indicate things like arithmetic overflow or underflow. These include 61 Add Subtract Multiply Divide Convert BCD to Binary Convert Binary to BCD -------------------- 62 Block Transfer Instructions Block Transfer Instructions are output instructions that are used to transfer up to 64 16 bit words of data between the data table and I/O modules. These are used with intelligent I/O that worked with words rather than individual bits. 63 Examples of these are things like Analogue I/O that read and write varying voltages rather than just on or off. PID that handle closed loop control of things like temperature. Thermocouples that read temperatures. Servo motor control that command positions of servo motors. Stepper motor control. Position encoders. 64 I won't go into details on block transfers, but this should be enough to give you a general idea of how the PLC could handle that sort of equipment as opposed to the simple on/off states of digital modules. Obviously these can be used in combination with GET, PUT, and other data manipulation instructions which can read and write memory locations. -------------------- 65 Jump Instructions and Subroutine Programming These include the following instructions. 66 Jump (or JMP) Label (or LBL) Jump to Subroutine ( or JSR) Return (or RET) 67 Jump will jump execution to a label position. You can think of it like a GOTO instruction. 68 Labels are numeric and can be from 00 to 77 in octal. Jump instructions are rung outputs, and when the rung goes true, execution jumps to the label number specified in the instruction. Labels are input instructions and appear at the start of the rung where execution resumes. 69 Normal jumps cannot jump into a subroutine, and they can only jump forward, not backwards. This means that loops are not possible using jump. This would be a deliberate design decision to prevent programs from getting stuck in a loop and preventing the scan from proceeding. 70 Subroutines appear after the end of the main program. They are defined by numerical octal labels using the label instruction. Subroutines are called using the Jump to Subroutine (or JSR) instruction. At the end of the subroutine, the Return (or RET) instruction returns execution to the next rung following the JSR which called it. You can nest calls to subroutines up to 8 deep. -------------------- 71 Data Transfer File Instructions Despite what you may think from the name, there are no disk files in the PLC2. A "file" instruction rather moves blocks of words from one data memory location to another. = These include source and destination addresses, and various other parameters which control the operation. There is actually quite a bit involved in these, but I'm not going to bother going into the details. You just need to know that it is possible to move blocks of data memory around. -------------------- 72 Shift Register Instructions These operate on entire series of words and include = Shift File Up Shift File Down FIFO Load FIFO Unload 73 These can operate on 1 to 999 words. A typical use for this sort of instruction would be to track a part as it moves through the various stations of a machine. -------------------- 74 Bit Shifts These operate on individual bits and include the following. 75 Bit Shift Left Bit Shift Right Examine Off Shift Bit Examine On Shift Bit Set Shift Bit Reset Shift Bit 76 These do pretty much as you would expect in terms of shifting bits left or right. However, the shift register can be defined to be anywhere from 1 to 999 bits long, so it is not confined to the size of a word. -------------------- 77 Sequencer Instructions Sequencer instructions are a software emulation of electromechanical drum controllers. Drum controllers allowed a long sequence of operations to be performed. These used a rotating drum which had a regular series of parallel holes in the outer surface. Above one row of these holes was a row of limit switches. By inserting pegs into some of these holes which made contact with the limit switches, the drum controller could turn on patterns of relays or valves. 78 Some of the limit switches could be connected to sensors on the moving elements of the machine. When the parts of the machine were in the correct position for that step in the sequence, the pattern of pegs would cause an electrical path to complete through the limit switches and energize a solenoid which would ratchet the drum to the next step. 79 This is often compared to a music box or to an old style of washing machine control. However in the case of the drum controller, the pegs would program the input match conditions as well as the desired output. The PLC/2 drum controller would emulate this sequencer. This feature came to be very widely used in equipment used in the automotive industry and other assembly oriented manufacturing processes and often formed the heart of the PLC/2 program. 80 I won't go into details of the sequencer instructions, but there are separate input and output sequence instructions. The input sequence instruction would try to match the states of the sensors, and the output sequence instruction would turn the various outputs on or off, controlling things like solenoid valves. The sequence itself would be defined by a series of words in memory, with each bit corresponding to an individual input or output. -------------------- 81 File Logic Instructions File Logic instructions amounted to array operations on words, although the term "file" is used for what in computer terminology would be an "array". These instructions included AND OR EXCLUSIVE OR Complement -------------------- 82 Special Programming Techniques These are miscellaneous instructions. These consist of the following. One Shot Leading Edge One Shot Trailing Edge 83 A one shot is an output instruction that causes the addressed bit to turn on for one scan upon a change of state in the rung logic condition. One shots could be triggered on a false to true change or a true to false change, depending on which of the two instructions were used. -------------------- 84 Why Did the PLC2 Use BCD? You are probably wondering why the PLC2 used BCD numbers instead of normal integers. This reflects the operating environment that an early PLC would have to work in. Early operator panels using mechanical numerical input devices and simple electronic output displays which typically worked in BCD. 85 Thumbwheel switches were small drum like rotating devices with numbers 0 to 9 printed on the periphery. Several wheels would be stacked beside one another. They looked somewhat like the mechanical odometer that you would find in an older car. The operator could however move each digit wheel separately. Each digit would be read by wiring 4 wires back to 4 inputs on the PLC. Thus each digit was inherently BCD due to how the thumbwheel hardware worked. 86 By turning the individual wheels separately, the operator could adjust things like temperatures, speeds, time delays, etc., and so modify process parameters. Output operator displays were typically 7 segment LED, neon, or vacuum fluorescent displays. These displays or devices similar to them preceded PLCs and so were common industrial devices that had to be handled. Each numeric digit had 4 inputs that would be wired to 4 outputs on the PLC. Seven segment displays can use hexadecimal, but that wasn't very useful if you are trying to display a temperature in Celsius to an operator. Thus again, BCD was the most practical sort of number to work with. 87 As PLCs and their user programs grew more capable and more sophisticated, they began working with normal integers and even floating point numbers. However, many if not most PLCs were to retain instructions to translate to and from BCD. -------------------- 88 The Influence of the PLC2 The PLC2 was a hugely successful product for Allen Bradley and firmly established them as one of the leading PLC vendors, with a number one position in the North American market, and in the top 5 world wide. However, it had a number of weaknesses, particularly the very primitive memory and subroutine architecture and the focus on octal and BCD numbers. It was focused on replacing control relays and it did this fairly well. 89 The next really successful successor to the PLC2 for Allen Bradley was the PLC5. This took the basic PLC2 instruction set and married it to a new memory system with new data types, separate name spaces, and parameterized functions. 90 It was like switching from programming in Basic with line numbers and jumps to Pascal. However, it could still use PLC2 I/O racks and modules, an important consideration given that the bulk of the cost of a PLC was in the I/O system. 91 The PLC 5 was in turn succeeded by the Contrologix series which modernized things still further, while being in a smaller form factor. Allen Bradley remain one of the major PLC vendors today, operating under the Rockwell name. -------------------- 92 Conclusion In this episode we focused on one specific but very widely used early PLC, the Allen Bradley PLC2. We covered the following subjects. The internal electronics. The data table. The user program. The T3 programming terminal. And a very brief overview of the instruction set. 93 This was a fairly simple and in some ways primitive PLC, but you should have a very rough idea of the concepts behind a PLC, including the data table and scan concept and the relay-like instruction set. 94 In the next episode we will take a look at a PLC from the largest vendor word wide, Siemens. In this example we will look at a model from slightly later than the PLC2, the S5 series. This could be seen as a second generation PLC. Since we won't need to repeat the basic concepts covered already, we can focus more on what is new and different about the S5 compared to the PLC2. 95 This has been the third episode in an 8 part series. -------------------- Provide feedback on this episode.
-
9
HPR4727: friedcamp interviews part 4: The free Lantern
This show has been flagged as Clean by the host. jasper from the Free Latern https://www.thefreelantern.com/ LyricConductor https://github.com/jnuyens/LyricConductor ModuleJail: https://www.modulejail.com https://github.com/jnuyens/modulejail Smart Dinosaur - Space for Fantasy Youtube: https://www.youtube.com/watch?v=qnUyW2yiiwo&list=OLAK5uy_kCrIROefktTWed3iPyDGhCUKl4gTvqXDU Spotify: https://open.spotify.com/album/3J2qdIgJFtk78O5IhYWbOH?si=eba8037fe0b34339 Provide feedback on this episode.
-
8
HPR4726: Trurl's intro show
This show has been flagged as Clean by the host. Introduction & Setting : Recorded on a sunny August afternoon in a Merton garden with Trurl exploring a wide-ranging philosophical and analytical journey through mathematics, games, and literature. Literary & Sci-Fi References : Stanislaw Lem : Discussion of the robot character Trurl from The Cyberiad , who alphabetically creates everything before reaching the letter "N" for "nothing" and destroying everything. Learn more about Lem on Wikipedia . Vladimir Nabokov : Analysis of The Luzhin Defense , a novel exploring chess obsession, psychological breakdown, and the "horror of chess." Explore the book on Wikipedia . Edgar Allan Poe & Thomas Hobbes : Discussion on 19th-century precision in writing, stylistic inflections, and the value of rigorous language. Martin Amis : Praise for his modern precision, with highlights including Money and Information . Core Philosophical & Mathematical Themes : Socratic Epistemology : Exploring the ancient Greek view from Plato's dialogues that equates knowledge directly with practical skill. Active vs. Empirical Learning : Trurl's theoretical aversion to traditional science labs in favor of pure conceptual frameworks, contrasted with hands-on computational tinkering. Stephen Wolfram : Discussion of A New Kind of Science and the computational approach to universal systems. Applied Puzzles & Strategic Games : Chess Ratings : Investigating the rating system, Elo mechanics, and statistical anomalies suggesting top players are statistically underrated. The Drake Equation & Astrobiology : Quantitative modeling of extraterrestrial life inspired by Stephen Webb's If the Universe Is Teeming with Aliens... Where Is Everybody? . Economics & Libertarianism : Transition from early socialist upbringing to decentralized economic structures, cryptocurrency architectures, and the Lightning Network. Provide feedback on this episode.
-
7
HPR4725: HPR Beer Garden - Tasting and reviewing beers
This show has been flagged as Clean by the host. In a bonus episode for the reserve queue, Kevie and Dave talk about how they review beers: Aroma - what do you smell from the glass and bottle or can Taste - take a sip, swirl it around your mouth and allow it rest a second before drinking; don't gulp. What flavours are you getting. Wait - What flavours are appearing in the aftertaste. Opinions - what are your personal thoughts on this beer. Score - pick a score and rarely give any beer full marks. Connect with the guys on Untappd: Dave Kevie The intro sounds for the show are used from: https://freesound.org/people/mixtus/sounds/329806/ https://freesound.org/people/j1987/sounds/123003/ https://freesound.org/people/greatsoundstube/sounds/628437/ Provide feedback on this episode.
-
6
HPR4724: HPR Beer Garden 19 - Lager
This show has been flagged as Clean by the host. As we are currently going through a warm patch (well, at the time of recording anyway), the HPR Beer Garden returns to provide some refreshment in the heat with a glass of cold and refreshing lager. Dave chooses Super Bock from Portugal, whilst Kevie picks City Limits Amber Lager from the tiny Dog Falls Brewing Co in Inverness. Connect with the guys on Untappd : Dave Kevie The intro sounds for the show are used from: https://freesound.org/people/mixtus/sounds/329806/ https://freesound.org/people/j1987/sounds/123003/ https://freesound.org/people/greatsoundstube/sounds/628437/ Upcoming beers: Mild Bitter Pale Ale Provide feedback on this episode.
-
5
HPR4723: friedcamp interviews part 3: The fried telephone system
This show has been flagged as Clean by the host. Lieven Blancke: https://mastodon-belgium.be/@lieven asterisk: https://www.asterisk.org/ Asterisk audio files: https://downloads.asterisk.org/pub/telephony/sounds/ Audio of PSTN call: <a href="https://en.wikipedia.org/wiki/File:1TR110-1_Kap8.3_Freiton1.ogg"> https://en.wikipedia.org/wiki/File:1TR110-1_Kap8.3_Freiton1.oggProvide feedback on this episode.
-
4
HPR4722: Show and tell: AI tropes
This show has been flagged as Explicit by the host. Obviously, I'll need to introduce why I'm the best person to explain this concept and how I did my research. After that, I dive into the core of he episode: the research. The Ultimate Guide To Spotting AI Generated Text That Every Writer Needs To Read. That title already has two tells: Title case : every word capitalised, which conveniently cannot be read out loud Hyperbole : a promise of ultimacy, for a ten minute podcast For most of this episode I am not myself, I am the machine, and every sentence I say is an example of the thing it describes. This page is the key to that part of the show: the tells in the order you hear them, with what they sound like in the recording. Most names come from tropes.fyi , so you can look them up there and get the longer explanation. As also stated in the podcast: these aren't errors. They are ordinary figures of speech that human writers have used for centuries. The tell is the density. Part one: Vocabulary Prestige vocabulary : words nobody uses out loud. e.g. delve, harness, utilise, robust, foster Grand nouns for ordinary things : e.g. tapestry, landscape, realm, ecosystem, paradigm Magic adverbs : imply significance, carry no information. e.g. quietly, fundamentally, arguably, remarkably Avoiding the verb "to be" : plain "is" swapped for something that sounds busier. e.g. serves as, stands as, represents, marks Invented concept labels : an abstract problem noun (trap, creep, paradox, inversion) welded onto a domain word, then used as if it were established. e.g. "the fluency trap", "prestige creep" Vague attributions : authority without a name. e.g. "experts agree", "industry reports suggest", "observers have noted" Appeal to familiarity : consensus claimed rather than proven. e.g. "a classic", "famously", "as we all know" Promotional register : describing turns into selling. e.g. unlocks, seamless, unprecedented Part two: Sentence shapes Negative parallelism : the "it's not X, it's Y" pattern, and the most reported tell of all. e.g. "This isn't a stylistic quirk, it's a structural signature." Not X. Not Y. Just Z. : the same move tripled, building tension for a reveal that needed none. e.g. "Not a habit. Not a tic. A tell." Sets of three : one tricolon is elegant, three in a row is a stuck needle. e.g. "it arrives in threes, it lands in threes, it exhausts in threes" Short punchy fragments : emphasis manufactured by full stops instead of earned by content. e.g. "Short. Parallel. Relentless." Self-answered questions : asks what nobody asked, answers it immediately. e.g. "The problem? Nobody asked." Anaphora : the same sentence opening until it becomes furniture. e.g. "They open sentences the same way. They open the next one the same way." Trailing "-ing" analysis : shallow significance bolted onto a finished sentence. e.g. "highlighting its enduring importance", "reflecting broader trends" False ranges : "from X to Y" where there is no scale and no middle. e.g. "from cesspit to citation" Filler transitions : connect nothing to nothing. e.g. "it's worth noting", "importantly", "notably" The interjection reflex : every sentence broken in half, whatever the punctuation. e.g. em-dash, double hyphen, comma pair, colon Comma-clipped tail : a short phrase hung off a comma instead of landing the point. e.g. "above the content, and save." Part three: Tone Manufactured suspense : the buildup arrives, the revelation does not. e.g. "here's the thing", "here's the kicker" Patronising analogy : teacher mode, uninvited. e.g. "think of it as a Swiss Army knife for condescension" Forced figurative language : a metaphor picked for cleverness rather than clarity. e.g. "Picture a bicycle. Now picture that the bicycle is prose." Stakes inflation : everything is historic. e.g. "a pivotal moment in the evolution of reading itself" False vulnerability : honesty performed, never risked. e.g. "let me be honest with you here. Genuinely honest." Assertion in place of evidence : e.g. "history is unambiguous on this point", "the evidence is clear" "Imagine a world where..." : the futurism sales pitch, pleasant outcomes if you accept the premise. e.g. "imagine a world where every paragraph carries a quiet intelligence" Quotable one-liners : slide bait that survives being pulled out of context because it never had any. e.g. "Every bite is a chance to be here, now, fully." The collaborative "we" : one author, plural pronoun. e.g. "we will unpack each one in turn" Sycophancy : not a figure of speech but a training artefact, and the fastest way to spot raw chatbot output. e.g. "What a genuinely profound question." Part four: Formatting Title case headings : see the top of this page Bullet lists where prose would do : the document turns into a slide deck. e.g. this page, from here to the bottom Bold-first bullets : every item opens with a bolded keyword and a colon. e.g. yes, also this page, I know Emoji as punctuation : rocket for aspiration, key for insight, sparkles because the paragraph ended Em-dashes : mid-sentence, twice, sometimes three times, where a comma would have done the job The find and replace tell : double hyphens everywhere, because someone removed the character and kept the habit Unicode decoration : arrows and curly quotes nobody typed on a keyboard. e.g. Input → Processing → Output One thought per line : paragraphs abolished. e.g. "It happened. Alone. On a Tuesday." Part five: Composition Announce then answer : says what the text is about to do instead of doing it. e.g. "This guide will explore what those signs look like." Rapid-fire historical analogies : authority borrowed from a list, with no argument underneath. e.g. "Marconi. Sony. Apple. Print, radio, web, mobile." The metaphor that will not end : e.g. the garden that stays a garden for six sentences, then becomes a north star that moves the needle Self-echo : a word from earlier returning as if it were being paid off, when it is the same narrow vocabulary surfacing again One point diluted : the same idea in five coats, sounding like progress. e.g. "put differently", "to rephrase", "in other words" Fractal summaries : every section summarises itself, including the one you are inside. e.g. "In this section we explored composition." The tie-back : closes by looping the answer back to the original question, long after the point was made. e.g. "So, to answer your question..." Concede and dismiss : e.g. "despite its fluency, it faces challenges typical of automated systems. Despite those challenges, it endures." Signposted conclusion : competent writing does not need to announce that it is ending. e.g. "in conclusion", "to sum up", "in summary" The conclusion that refuses to end : clause stacked on clause, quieter and more meaningful each time. e.g. "the real AI generated text was the paragraphs we scrolled past along the way" Engagement bait : e.g. "Agree?" plus a pointing emoji Tells on the list that did not fit in the episode The recording would have run to forty minutes. These are worth knowing anyway. Reasoning leak : the text narrating its own decisions instead of just making them. e.g. "I want to be exact about my own role here." Premise stacking : a paragraph of evidence in front of the point, so the point has been made twice before it arrives Compulsive counting : states the number of items before listing them. e.g. "Five things we wish to discuss" Excessive enumeration : a listicle disguised as prose. e.g. "The first wall is... The second wall is..." Belaboring the unnecessary : defending a minor point against an objection nobody was going to raise Synonym cycling : refusing to repeat a noun. e.g. the dashboard, then the interface, then the portal, then the analytics hub Wh-headings : the default shape a model reaches for when naming a section. e.g. "What this means", "Why it matters" Content duplication : the same paragraph appearing twice, because the model lost track of what it had written Why this is not a detector guide Liang, Yuksekgonul, Mao, Wu and Zou, Stanford, published in Patterns (Cell Press) on 10 July 2023. Seven commercial GPT detectors were run over 91 TOEFL essays written by non-native English speakers and 88 US eighth grade essays. Average false positive rate on the TOEFL essays: 61.3 percent. 89 of the 91 essays were flagged as AI generated by at least one detector. All seven agreed on 18 of them. The US eighth grade essays came back near perfect. Plainer word choice, lower perplexity, reads as machine to the software. Feeding the TOEFL essays through ChatGPT with the prompt "enhance the word choices to sound more like that of a native speaker" dropped the false positive rate to 11.6 percent. So the way to stop being accused of writing like a machine was to have a machine rewrite you. Do not take my word for it, the paper is open access: https://www.cell.com/patterns/fulltext/S2666-3899(23)00130-7 Preprint: https://arxiv.org/abs/2304.02819 The pointers, for real Write with things in it: names, dates, prices, the model number of the drive that died. Say something a machine has no way to know, because it happened to you. Be willing to be wrong out loud. Keep one weird sentence that your editor would cut. Links and references The list this episode leaned on: tropes.fyi , by ossama.is The detector study: Liang et al., "GPT detectors are biased against non-native English writers", Patterns, 2023 "Not an Addict", K's Choice, 1995. It's not a habit, it's cool, I feel alive. "Friends, Romans, countrymen" and "I come to bury Caesar, not to praise him": Shakespeare, Julius Caesar, act 3 scene 2 "Veni, vidi, vici", attributed to Julius Caesar by Suetonius and Plutarch Over to you There is a comment section under this episode. Tell me which tell I missed, and which one you are guilty of. Better yet, record it. Read your worst, most machine sounding paragraph out loud and post it to Hacker Public Radio. The queue is always hungry. Provide feedback on this episode.
-
3
HPR4721: HPR Community News for August 2026
This show has been flagged as Explicit by the host. New hosts Welcome to our new hosts: Kristoff number9 Last Month's Shows Id Day Date Title Host 4696 Mon 2026-08-03 HPR Community News for July 2026 HPR Volunteers 4697 Tue 2026-08-04 Correcting the Dates of Files Vance 4698 Wed 2026-08-05 ID3 Tags and Vorbis Comments Whiskeyjack 4699 Thu 2026-08-06 Sunshine, Moonlight, Playnite !? operat0r 4700 Fri 2026-08-07 Robert A. Heinlein: The Juveniles Ahuka 4701 Mon 2026-08-10 Elsbeth muses with Lee on her experiences and opinions on AI Elsbeth 4702 Tue 2026-08-11 MakeACopy is an open-source document scanner app for Android Ken Fallon 4703 Wed 2026-08-12 Ebooks and Audiobooks Ahuka 4704 Thu 2026-08-13 HPR Beer Garden 17 - Double DDH IPAs Kevie 4705 Fri 2026-08-14 Free Software Is Wasted On You Teens Trollercoaster 4706 Mon 2026-08-17 Quirks and Customisations Lee 4707 Tue 2026-08-18 UNIX Curio #12 - expr Vance 4708 Wed 2026-08-19 Programmable Logic Controls - Episode 1 Whiskeyjack 4709 Thu 2026-08-20 Peertube: my two penneth Kevie 4710 Fri 2026-08-21 Civilization VI Ahuka 4711 Mon 2026-08-24 Second-Hand Phone TrumpetJohn 4712 Tue 2026-08-25 friedcamp interviews part 1: Wim van Gool Kristoff 4713 Wed 2026-08-26 July2026 Community News response, and Future Plans TheDUDE 4714 Thu 2026-08-27 HPR Beer Garden 18 - Amber Ale Kevie 4715 Fri 2026-08-28 friedcamp interviews part 2: Thomas Farstrike Kristoff 4716 Mon 2026-08-31 PocketMod TrumpetJohn Comments this month Past shows hpr3708 (2022-10-19) "Insomnia as a Hobby" by TrumpetJohn. Robert St.Preux said: "Insomnia as a hobby" (2026-08-30 00:56:59) hpr3766 (2023-01-09) "ACER Nitro 5 laptop review" by Bookewyrmm. wevaba6950 said: "A refreshing contrast between enterprise gear and low-power hardware" (2026-08-10 09:19:47) candycanearter07 said: "re: a refreshing contrast..." (2026-08-12 02:54:05) hpr4581 (2026-02-23) "Sharp Intake of Breath City (A.K.A.) How I learnt to stop worrying about the fork bomb" by not_toby. Windigo said: "Fantastic episode" (2026-08-08 16:51:00) hpr4655 (2026-06-05) "Tips for Glasses" by norrist. Archer72 said: "Glasses and age" (2026-08-06 11:41:39) hpr4676 (2026-07-06) "HPR Community News for June 2026" by HPR Volunteers. Archer72 said: "hpr4657 :: UNIX Curio #8" (2026-08-02 19:29:26) hpr4677 (2026-07-07) "UNIX Curio #10 - Checksums and Hashes" by Vance. Whiskeyjack said: "Checksums and HPR4677" (2026-07-31 00:37:40) hpr4678 (2026-07-08) "High Resolution Elapsed Time in Shell Scripts" by Whiskeyjack. Archer72 said: "Reply to Whiskeyjack and candycanearter07 " (2026-08-03 10:12:47) hpr4692 (2026-07-28) "Noise Music Tutorial 2: Using Audacity to Make Noise" by TheDUDE. Windigo said: "Audacity for noise music" (2026-08-06 16:09:59) candycanearter07 said: "re: Audacity for noise music" (2026-08-12 18:55:18) This month's shows hpr4696 (2026-08-03) "HPR Community News for July 2026" by HPR Volunteers. Archer72 said: "Amateur Radio Field Days" (2026-08-02 11:50:11) Archer72 said: "hpr4673 First contact" (2026-08-03 10:16:09) candycanearter07 said: "re: mic issues" (2026-08-03 22:21:42) Th3Dud3 said: "response to Kevin not being able to get cream ale" (2026-08-06 00:12:38) hpr4697 (2026-08-04) "Correcting the Dates of Files" by Vance. Jim DeVore said: "Useful episode!" (2026-08-04 10:53:12) Vance said: "Modifying metadata" (2026-08-07 01:44:51) Henrik Hemrin said: "Exif in digiKam" (2026-08-08 19:20:13) candycanearter07 said: "useful!" (2026-08-10 16:08:18) hpr4698 (2026-08-05) "ID3 Tags and Vorbis Comments" by Whiskeyjack. xmanmonk said: "Great episode!" (2026-08-05 15:19:01) candycanearter07 said: "quirks in hpr" (2026-08-05 15:52:18) Whiskeyjack said: "Reply to xmanmonk in HPR4698 - ID3 Tags" (2026-08-06 00:12:26) Whiskeyjack said: "Reply to candycanearter07 in HPR4698 ID3 tags" (2026-08-06 00:28:48) hpr4703 (2026-08-12) "Ebooks and Audiobooks" by Ahuka. candycanearter07 said: "useful!!" (2026-08-12 02:30:58) Archer72 said: "Audiobooks and Audible" (2026-08-12 09:06:40) Jack said: "Misleading" (2026-08-12 17:05:21) hpr4705 (2026-08-14) "Free Software Is Wasted On You Teens" by Trollercoaster. Archer72 said: "Brilliant! " (2026-08-03 14:08:08) Trollercoaster said: "One left in my six pack" (2026-08-14 07:53:38) old guy said: ":-)" (2026-08-14 09:24:15) Trollercoaster said: "old guys together" (2026-08-14 13:14:48) candycanearter07 said: "good satire again" (2026-08-14 21:42:55) Trollercoaster said: "Who's talking about satire?" (2026-08-15 07:54:33) Trollercoaster (our of character) said: "Woopsie" (2026-08-16 19:25:41) hpr4706 (2026-08-17) "Quirks and Customisations" by Lee. candycanearter07 said: "cool ep!" (2026-08-30 20:15:40) hpr4707 (2026-08-18) "UNIX Curio #12 - expr" by Vance. xmanmonk said: "Another great show" (2026-08-18 22:22:56) Whiskeyjack said: "HPR4707 Unix curio - dc" (2026-08-19 14:31:02) Vance said: "dc" (2026-08-20 04:47:32) Whiskeyjack said: "HPR4707 Unix curio - Stack Machines" (2026-08-21 16:00:48) Trollercoaster said: "Suggestion for Linux Curio" (2026-08-27 10:42:43) Whiskeyjack said: "Obscure X Window programs" (2026-08-27 16:16:15) candycanearter07 said: "re: Suggestion for Linux Curio" (2026-08-27 18:13:51) Archer72 said: "re: Suggestion for Linux Curio" (2026-08-28 10:07:31) Vance said: "Thanks for the ideas" (2026-08-28 16:39:28) Whiskeyjack said: "Reply to Vance on episode research" (2026-08-30 04:14:01) candycanearter07 said: "re: re: Suggestion for Linux Curio" (2026-08-30 12:06:31) hpr4708 (2026-08-19) "Programmable Logic Controls - Episode 1" by Whiskeyjack. Archer72 said: "Thanks you" (2026-08-08 10:49:34) Whiskeyjack said: "Reply to Archer72 on HPR4708 - PLCs Episode 1" (2026-08-08 17:11:10) Trey said: "Memories of Ladder Logic" (2026-08-19 13:01:41) Whiskeyjack said: "Reply to Trey on HPR4708 - PLCs Episode 1" (2026-08-20 02:02:44) Trey said: "Cyber in ICS" (2026-09-02 22:31:51) Whiskeyjack said: "Reply to Trey on Cyber in ICS" (2026-09-03 15:24:51) hpr4709 (2026-08-20) "Peertube: my two penneth" by Kevie. xmanmonk said: "Thanks for the episode" (2026-08-20 19:15:57) hpr4711 (2026-08-24) "Second-Hand Phone" by TrumpetJohn. candycanearter07 said: "interesting!" (2026-08-24 19:17:44) Whiskeyjack said: "HPR4711 - TCL Flip phone" (2026-08-24 22:22:11) hpr4712 (2026-08-25) "friedcamp interviews part 1: Wim van Gool" by Kristoff. Archer72 said: "Interview" (2026-08-26 23:11:49) Laindir said: "Fascinating interview!" (2026-08-26 23:18:33) hpr4716 (2026-08-31) "PocketMod" by TrumpetJohn. Kightlinger said: "I love PocketMods!" (2026-08-31 17:35:43) candycanearter07 said: "analog solutions" (2026-08-31 20:52:55) Mailing List discussions Policy decisions surrounding HPR are taken by the community as a whole. This discussion takes place on the Mailing List which is open to all HPR listeners and contributors. The discussions are open and available on the HPR server under Mailman. The threaded discussions this month can be found here: https://lists.hackerpublicradio.com/pipermail/hpr/2026-August/thread.html Events Calendar With the kind permission of LWN.net we are linking to The LWN.net Community Calendar. Quoting the site: This is the LWN.net community event calendar, where we track events of interest to people using and developing Linux and free software. Clicking on individual events will take you to the appropriate web page. Provide feedback on this episode.
-
2
HPR4720: Robert A. Heinlein: Earlier Novels
This show has been flagged as Clean by the host. Heinlein published his first novel in 1948, and within a decade had published some major works. What distinguishes these early novels from his later ones is that he has not pushed the boundary of what he can write about to the degree he does in the 1960s and later. But there are fine novels in this period. Heinlein’s Earlier Novels During the 1950s Heinlein was busy with his Juvenile series, but also found time to write some adult novels, and some of them are very good. A few of them were expansions of earlier, shorter works that he turned into full novels. Beyond This Horizon (1948) – This is Heinlein’s first novel after the lackluster For Us The Living (which never was published during his lifetime; Heinlein knew it was sub-par). It is set in a future society that is practicing a form of eugenics, with a breeding program designed to produce the perfect human by selecting for health, longevity, and intelligence. The protagonist, Hamilton Felix, is not the one, but he is near to the end. And this society has also made it a point to keep a certain number of people who are not part of the breeding program, as a form control for the experiment. Hamilton Felix is in fact the penultimate step in this program, but has so far shown no interest in marriage or procreation. But in the course of the novel he is persuaded. His children are shown to be the end point of the program at the end of the novel. A sub-plot involves his friend Mordan Claude (in this book everyone puts their family name first) who falls for a woman he meets. When she sneezes, he begins to suspect she is one of the “control naturals”. They are protected, but clearly this is a class society, which is to be expected in any society that practices eugenics, and should be off-limits for Claude. But love conquers all, and then he discovers that she is not a control natural after all, which to my mind is a somewhat weaker outcome.Another aspect of this society is that most people are armed, and dueling is common. This is explained as “An armed society is a polite society,” since if you offended someone you could find yourself in a duel very quickly. A lot of gun enthusiasts have picked up on this and repeated it as if it was revealed truth, but even a cursory examination of societies with a lot of guns shows it to be false. Sixth Column (1949) – This novel was originally a serial written in 1941, when the Sino-Japanese war was in full swing, and this motivates some of the ideas in this book. The setting is a United States that has been conquered by the Pan-Asian Federation. This Federation had previously absorbed the Soviet Union and India, and is portrayed in very much Yellow Peril manner. John W. Campbell had a lot to do with this, and it makes the novel something that you can skip without losing much. The Pan-Asians regard the Americans as slaves. And their policy towards their slaves is “Three things only do slaves require: work, food, and their religion.” In an American military outpost hidden in the mountains of Colorado a miracle weapon is found that can kill, and it can somehow be tuned to only affect the Asians and not the Americans. Since the Pan-Asians have allowed religion (better to pacify the slaves), the Americans create a religion and use that to infiltrate the Pan-Asian facilities. Heinlein himself had a low opinion of this novel, and deservedly so. The Puppet Masters (1951) – This novel is a product of the paranoia of the times regarding Communists. But this novel is more of an allegory than a direct anti-Communist screed. It is fundamentally about “the enemy within”, in the form of alien parasites called slugs that can take over a human by controlling the nervous system, making the human a powerless slave to the alien. And they are attempting to conquer the world, though all of the action we see takes place in the United States. This is essentially The Body Snatchers plot, and has been done many times. But both this novel and the previous Sixth Column reflect the feeling of paranoia prevalent in society when people thought there were communists under every bed looking to corrupt the youth and subvert The American Way of Life. And it is notable that the organization that leads the fight against the slugs is a top secret outfit established without the knowledge of Congress and responsible only to the President. Double Star (1956) – This is when Heinlein as a novelist starts to really click, and it was shown by his winning his first Hugo for best novel. The plot is somewhat reminiscent of Prisoner of Zenda. In a future solar system, Earth, Venus, Mars, and the Jovian satellites are united under a parliamentary democracy, which is in turn under a constitutional monarchy. Think of it as similar to the UK, only in this case it is the House of Orange that are the monarchs. A politician named John Joseph Bonforte is the leader of the Expansionist coalition, and one of the most prominent politicians in the solar system. And he has been kidnapped sat a crucial time. His staff find an actor, stage name The Great Lorenzo, who is similar enough physically to impersonate Bonforte, while they try to get the real Bonforte back. They do succeed, but he has been very badly treated by the kidnappers, so the actor has to continue for the election campaign. And he wins the election, but the real Bonforte dies. The actor then continues to impersonate him to the point of really becoming him. The two things that make this novel so interesting are first, the focus on acting as seen through the eyes of Lorenzo; and second, the focus on the machinery of politics. And this draws upon Heinlein’s own experience in electoral politics. Though he was not successful in getting elected, he clearly had absorbed a lot about the how-to of the process. And he did write a non-fiction book called Take Back Your Government (1992), which was published posthumously. It was subtitled A Practical Handbook for the Private Citizen Who Wants Democracy to Work, and was originally written in 1946. It was based on his experience with EPIC and in trying to get Upton Sinclair elected Governor of California in 1934. The Door Into Summer (1957) – This is another good novel, which features several themes that would reappear through Heinlein’s work. The first is that Heinlein was a very confirmed cat lover. The title comes from a cat who went from door to door in the middle of winter, trying to find the door into summer. Second, the protagonist is an engineer, which is of course Heinlein’s background. The third is the time travel element. People can travel into the future through a type of suspended animation called Cold Sleep. But for the purposes of the plot a scientist has invented a time travel device that sort of works. It sent a grad student named Leonard Vincent a few centuries back in to Italy where he became Leonardo da Vinci. But it sends our hero, Daniel Boone Davis, back exactly where he wanted to go. We see a few things in this novel that are of interest. First, the cat, Pete, has the full name of “Petronius the Arbiter”, who was the author of the Satyricon, so Heinlein’s interest in sex is starting to make an oblique appearance. Second, when Dan goes back in time, he lands in the middle of a nudist camp, and that is also something Heinlein was very interested in.The plot can be summarized as Dan gets robbed of his inventions by his business partner and his secretary/fianceé, gets put into Cold Sleep, then goes back in time to just before he got robbed and fixes things to his satisfaction, then takes the Cold Sleep again to finish his plans. It is a pretty good yarn. Methusaleh’s Children (1958) – This was originally a novella written in 1941, then expanded into a novel. It is part of the Future history. It introduces Lazarus Long, who would go on to be Heinlein’s alter ego through many of his future books, and thus implicitly makes those books part of the Future History, though most scholars will dispute this. What cannot be disputed is that this novel is a cornerstone for where Heinlein would go in many of his future stories.The story begins with a successful businessman named Ira Howard who dies of old age at around the age of 49 in the 19th century. In his will he leaves his estate to trustees with the mandate to find a way to extend human life span. The best they can come up with is to encourage (financially) people who had long-lived grandparents to marry each other and have children. They carry this on and by the 22nd century the Howard Families, as they call themselves, have an average life span of 150 years. But they have to hide this form the rest of society, and arrange from time to time to “die” and then become someone else. This masquerade helped them survive the years of Nehemiah Scudder and his successors, but a few of them reveal themselves during the Covenant government. This has a bad result because people want the secret that they assume the Howard families are hiding, and they might be willing to kill to get the secret. So the Howard families steal a starship and make their escape. One of the people onboard is Andrew Jackson “Slipstick” Libby, who we first met in the short story Misfit, and he manages to come up with a Faster-Than-Light drive. They finally decide to go back to Earth, where they discover that scientists have found ways to prolong human life and they are no longer enemies, in fact they are welcomed for bringing the FTL drive.This is definitely a must-read novel for anyone who wants to get into the Heinlein universe. So many other works are based on it. Links https://en.wikipedia.org/wiki/Beyond_This_Horizon https://en.wikipedia.org/wiki/Sixth_Column https://en.wikipedia.org/wiki/The_Puppet_Masters https://en.wikipedia.org/wiki/Double_Star https://en.wikipedia.org/wiki/Take_Back_Your_Government https://en.wikipedia.org/wiki/The_Door_into_Summer https://en.wikipedia.org/wiki/Methuselah%27s_Children https://www.palain.com/science-fiction/the-golden-age/robert-a-heinlein/heinleins-earlier-novels/ Provide feedback on this episode.
-
1
HPR4719: How to use a multimeter, introduction and basic usage
This show has been flagged as Clean by the host. Outline of this radio program: Short history of multimeters Differences between VTVM, VOM, DMM Why you may need a VTVM Analog vs Digital Current manufacturers Why you may want to use an analog meter Safety and CAT Probes Functions of the modern DMM and notes Wrap up Provide feedback on this episode.
-
0
HPR4718: Programmable Logic Controls - Episode 2
This show has been flagged as Clean by the host. -------------------- 01 Introduction This is the second episode in an 8 part series. 02 In the previous episode we discussed the predecessors of PLCs, in particular relay logic. In this episode we will discuss how relay logic came to be expressed in software rather than in actual hardware components. 03 The topics to be covered include * Early computers in industry. * The first PLC. * PC versus PLC - what's in a name. * Who the major brands are. * What does a PLC actually look like. * Machine architecture. * PLC programs. * The scan. * PLC programming languages. * Relative popularity of PLC programming languages versus more conventional computer programming languages. -------------------- 04 Early Computers in Industry Computers came to be used in industry fairly early on. Minicomputers were used in some large industries to control things like electric power plants. For example, the DEC PDP8 was used to control the refuelling machines in CANDU nuclear power plants. These would load and unload fuel from the reactor, which happens continually on a daily basis while the reactor is running. 05 The first microprocessor based computer sold on the commercial market was the MICRAL from France, based on the Intel 8008. It was sold as a cost effective replacement for minicomputers used in industry. It preceded what are generally considered to be the first Personal Computers. So you can see that industry were not reluctant to adopt new technology. 06 However, mini computers were large complex systems that did not fit well into a factory floor. They had particular niches in very large complex integrated systems, but were not suited to controlling many individual machines in a factory that produced things like automobiles or appliances. 07 What was needed was something that would fit into a standard electrical enclosure, withstand the temperatures found in a factory, could stand up to vibration and noise, was tolerant of voltage fluctuations, interfaced directly to sensors and actuators, and could be readily programmed by engineers, technicians, and tradesmen who were familiar with the processes to be controlled, but had little or no experience with computers. 08 This required a complete integrated package covering hardware, software, and product distribution through industrial supply retailers. Something that could meet these criteria is what was needed to become the PLC. -------------------- 09 History Many histories point to a US developer that became Modicon. However, their first hardware was more a proof of concept than a viable commercial product. In fact multiple companies in the US, Europe, and Japan were working on the problem and released comparable products all at around the same time in the early 1970s. 10 The idea preceded the implementation by a number of years. For example, General Motors had asked industrial control equipment suppliers in the mid 1960s for some sort of programmable control device to replace relay logic. This showed potential suppliers that there was a market for this sort of thing, and gave them a clearer idea of what customers were looking for. 11 What made this possible was the development of the first 8 bit microprocessor in that time period, combined with readily available bit slice processors from the minicomputer industry and other components. 12 People knew what to do, the problem was waiting for the development of suitable component hardware. Minicomputers were already being used in industry, but were normally located in control rooms. PLCs were an effort to take that technology out of control rooms and put it on the shop floor. 13 PC Versus PLC - What's in a Name In the early days, these systems were known as either PLC, which means "Programmable Logic Controller", or PC, which means "Programmable Controller". Different vendors favoured different terminology, but both were common. I have a book which was at one time one of the standard reference handbooks for this industry which was published in 1989, and refers to them as Programmable Controllers, or PCs. 14 The term PC was also used to mean "Personal Computer", but that wasn't really a problem in the early days. However, a certain large company decided to call their entry into the personal computer market the "IBM PC", and suddenly "PC" became a generic term for desktop computers. 15 After a long struggle to keep referring to their products as "PCs", even the biggest vendors caved in and gave up the fight and switched to using the "PLC" term. I will therefore use the term "PLC" in this podcast series even when talking about products which were originally called "PCs" at the time they were introduced. 16 Who the Major Brands Are The companies that came to dominate the industry fairly early on were generally companies that already made industrial electrical hardware. These were Siemens Allen Bradley, later known as Rockwell Schneider Mitsubishi Omron 17 These are still the dominant companies in the business, although there are many small brands, particularly at the cheaper end of the market. All of them were suppliers of a wide range of industrial control hardware, such that you could build all or nearly all of the parts of your control system using only their products. 18 Each of these sells an integrated hardware and software package, including development software, that is completely proprietary. If you thought that the mainframe business had a lot of vendor lock-in, you haven't seen the industrial market. -------------------- 19 What Does One of These Things Actually Look Like? At this point you are probably still very confused as to what a PLC actually is. I will attempt to describe one in basic terms by giving an example of one. The smallest and simplest ones are what are termed "shoebox" PLCs. These are basically a rectangular box with a plastic case. 20 Here are the dimensions for a typical low end model, a Mitsubishi FX3U-32M. According to specs in the manual, it is 150mm wide, 90mm high, and 86mm deep. 21 It can be mounted to the panel in an electrical enclosure by snapping it onto what is termed a DIN rail. DIN rails are standard mounting systems which use a strip of metal which is U shaped with a lip at the top of each end of the U. You screw the DIN rail to the electrical panel in the enclosure, and most industrial components such as PLCs, terminal blocks, circuit breakers, and all sorts of other things will simply snap onto it and be ready to wire together. 22 Along the top and bottom of the PLC are terminals to which you can attach wires from the things you want to sense or control in the rest of the machine, such as push buttons, pilot lights, proximity sensors, and valves. 23 Inputs are along the top, and outputs are along the bottom. The FX3U-32M has 16 inputs and 16 outputs. These I/O can be 24 volts DC, or 100 to 120 volts AC, depending on the model. Alternatively it may have small relays for outputs. While AC I/O once predominated, 24VDC became the standard in most industries several decades ago as it interfaced with electronic devices more readily and also allowed for cheaper and more compact wiring. 24 More inputs and outputs can be added by connecting additional I/O modules to the main PLC next to it on the same DIN rail, up to a total of 256 I/O The connection is typically via short ribbon cables which plug into the adjacent module. These I/O modules look like the main PLC, but just add I/O. 25 Inside the PLC are a CPU, memory, and firmware. This model has 64K "steps" of memory, which can be thought of as how many instructions you can have. The program runs in RAM, but you can add a small flash memory device to save the program to. There is a run/stop switch that allows you to start or stop the user program. There is a port that you can use to connect the PLC to a laptop computer with a cable so you can download your program to it. 26 The biggest part of the market for PLCs is for these "shoebox" style. However, there are bigger ones as well which have more I/O, more memory, faster CPUs, etc. These are meant for tasks such as coordinating large assembly lines and things like that. These are what are termed rack systems, where the rack is an empty box which is open at the front and has a backplane bus running along the back. You plug the main CPU into the bus, normally in the leftmost position, and then plug I/O modules into the rack. The I/O modules are tall, thin, fully enclosed boxes with the I/O terminals along the front. 27 You should have a rough idea of their appearance at this stage, so I'll leave the physical description aside for now and go on to the concepts behind how they actually work and what makes them different from something like say a Raspberry Pi. -------------------- 28 Machine Architecture It is not really the hardware which defines the PLC. Rather, it is the software system architecture. Every PLC that I am aware of has a common set of features. 29 Data Table A data table is simply a block of memory which may or may not be subdivided into different parts. All logic, and all, or nearly all, I/O works by writing to and reading from addresses in the data table. 30 I/O Every input and output point maps to a fixed memory address in the PLC. To read the state of an input, you read its memory address. To change the state of an output, you write to its memory address. 31 These digital I/O appear as single bit boolean values. The PLC programming language has instructions which allow single bits to be read or written to directly without have to mask off other bits as you would have to do if you were using a conventional programming language. I have so far only mentioned digital I/O, which are single bit on/off values. 32 Other types of I/O Many PLCs also have other types of I/O such as Analogue I/O, which represent variable voltages rather than just on/off values. For example you may wish to read a temperature from a thermocouple where the voltage level varies with the temperature. Often these are mapped into the data table as byte or word values. 33 Internal Bit or Boolean Memory There will be a range of single bit or boolean memory which is used for internal logic state. Your program may need to come up with a series of intermediate logical values, and you store them in these internal bits or flags. 34 Timers and Counters Control of machinery often involves timing and counting. Timer and counters are mapped into the data table. Timer and counter presets and values can be read as words, and there will be bit addresses which indicate the status of the timer or counter, such as whether it has reached its preset and is "done". 35 Integer and Floating Point Values There will be word addresses where integer and floating point words can be stored. If you need to do some mathematical calculations and store the results, you would save them in an integer or floating point word. 36 Typed Memory Unlike when programming something like a PC where you simply have a range of undifferentiated bytes and it is up to your software to impose meaning on it, PLC data table memory has defined types and meanings and the system firmware enforces the correct memory access methods. 37 Size of Data Table Data tables vary greatly in size. A bigger data table allows for bigger and more complex program. Generally, new PLCs will have bigger data tables than older ones, and more expensive PLCs will have bigger data tables than cheaper ones of the same generation. 38 Example Having picked a current PLC as an example for physical dimensions, I will pick an old and obsolete one for an example of a data table. The Siemens S5-100 series was a small PLC that came in several sizes. The basic S5-100 had the following data table size. 39 Digital I/O had a maximum of 128 inputs and outputs taken together. Analogue inputs and outputs had a maximum of 8 taken together. Flag, or bit, memory was 1024 Timers - 16 Counters - 16 40 The S5-103 offered more of everything in the same basic package, but of course at a higher cost. Digital I/O had a maximum of 256 inputs and outputs taken together. Analogue inputs and outputs had a maximum of 32 taken together. Flag, or bit, memory was 2048 Timers - 128 Counters - 128 -------------------- 41 The PLC Program A PLC will come with what amounts to an operating system and virtual machine built into it. You as the programmer use the vendor's proprietary development software, what is generally called the "PLC programming software" to write an application for it. 42 You then connect your PC to the PLC using a cable of some sort, and download your program into it. The program gets stored in the PLC's RAM. There may be flash memory which serves to back up the RAM when the power is off so you don't lose the program. Before flash was available, a battery was typically required to hold the static RAM memory. 43 There is just one program in the PLC. There is no file system, just the program memory and the data table. When the PLC starts up, it runs the program that you wrote. It continues running the program until you turn off the power or you set the run/stop switch to the stop position. 44 I will come back to programming later, but we needed to cover these basic points before I could describe some further concepts we need to cover. -------------------- 45 The Scan A key thing to understand about PLCs is the "scan" concept. This works as follows. There is a repeated cycle called the scan. 46 First, the PLC reads all the physical inputs and updates the corresponding input addresses in the data table. Next it runs the user program once. Finally it reads the output addresses in the data table and writes them to the corresponding physical outputs. Then it repeats the scan from step 1. 47 A scan can take anywhere from a few milliseconds to hundreds of milliseconds, depending on the model of PLC. Older PLCs and cheaper PLCs tended to be slower. However, as integrated circuit technology advanced, programs got faster. 48 The PLC has a watchdog timer. This is a timer which runs in the background and resets at the start of a scan. 49 If a scan takes too long, the watchdog will trip and stop the program and set the outputs to some defined state, either turning them all off or holding them at the last value. This is known as "faulting" the processor. 50 A watchdog fault may be caused by a program that is too long, or uses a lot of very slow instructions, or in some cases it may be due to a bug in your program. However, this last cause is not as common or as easy to cause as you may think when it comes to faults. -------------------- 51 PLC Programming Languages The key to all of this is the unique way in which the programming languages used in PLCs work. I will first briefly describe the two main programming languages and then explain how they work in a PLC as part of a complete system. 52 Ladder Logic There are several programming languages, but the main one is called ladder logic. If you listened to the previous episode, the word "ladder" will ring a bell. When machines were controlled by electromechanical relays, the electrical drawings which specified and documented the types of relays and the connections between them were called "ladder diagrams". 53 PLCs simply take these ladder diagrams and reproduce them on your computer screen in the programming software using the same standard schematic symbols. 54 You do not need to draw the symbols like it was CAD. Instead you simply use your keyboard or mouse to say that you want this symbol to be entered in the current cursor location or you want this wire to be connected from here to there. When your rung is complete the software will check at some point if it is syntactically correct. You then go on to entering the next rungs one after another until you have your complete program. 55 Instruction List An alternative representation is something called "instruction list", although various vendors may use other terms. This is a text representation that looks like a sort of assembly language. However, it's actually really ladder, just shown in a different way as text and some types of programming software will allow you to toggle between ladder and instruction list mode, translating between them automatically. 56 This should give you a clue as to how a PLC can execute a schematic diagram. Behind the scenes the programming software will translate the diagram into instruction list, and then the PLC will either execute those instructions in an interpreter, or compile them to machine code and execute those. 57 Other Programming Languages There are other programming languages, but they are rarely seen. I won't go into detail in terms of describing them, just briefly mention them. 58 One is called Sequential Function Chart. This is a type of flow chart which is designed to show complex sequences, including ones with alternate or parallel paths. The original name for this is Grafcet, spelled G R A F C E T. This stands for "Graphe Fonctionnel de Commande Étape Transition". As well as being a programming language, it is also a very good design and analysis tool even if the PLC being used doesn't support it. 59 Another is Function Block Diagram. This is another graphical language which resembles flow diagrams used in process industries. This is supposedly used mainly in industries such as chemical manufacturing. However it appears to be very niche and most people who use PLCs will never have seen it, and many will not have even heard of it. 60 Yet another language is Structured Text. This is very obviously derived from Modula-2 and strongly resembles it. If you are not familiar with Modula-2, it was created by Nikolas Wirth and intended as the successor to Pascal, which it was derived from. Structured Text seems to be much loved by a small number of academics, but seems to have little actual use in the field. You lose pretty much all of the monitoring and debugging facilities built into a PLC if you use it, so it's kind of pointless except perhaps for a few niche applications. There are a few other languages as well, mainly proprietary ones, often using flow charts or the like. 61 PLC Languages are Non-Blocking The important thing about a PLC program is that it executes from top to bottom, left to right, without stopping or blocking. There is no waiting on input or output or for a system call to return. Each rung immediately yields a result which is written to a memory address or which enables a timer or counter. Every rung is executed every scan. 62 You can reproduce this in a traditional computer programming language. In ladder logic it is inherent to the syntax and it is either very difficult or impossible to do it any other way. I'll give an example in Python of what I mean. 63 a = (b or c) and not d 64 If b or c are true and d is false, then a is set to true. If b and c are false or d is true, then a is false. 65 Now imagine that we are not executing this statement once, but rather are executing it repeatedly. This statement will never block execution. It will always immediately yield a result. Now let's modify that a bit. 66 a = (b or a) and not d 67 Note that we have replaced c with a. Now the value of a depends the previous value of a as well as b and d. However, once a becomes true, the value of b no longer matters because b is in an or condition with a. Now if we execute it over and over again, only d becoming true will make a go false. 68 This is a standard push button circuit, also known as a "seal in circuit", because it "seals" around the start condition. 69 b is the start push button with a normally open contact. d is the stop push button with a normally closed contact. a is a relay with one of its contacts being used to hold it on. 70 If you wired this up with actual push buttons and relays or if you programmed it into a PLC with ladder logic it would work the same way. A PLC program will consists of rung after rung of logic like this and executes it repeatedly scan after scan, only pausing to update its I/O. 71 Async Programming Some of you may be thinking that this cyclical scan sounds like the async programming that is all the rage with web server applications these days. Essentially it is exactly the same principle. 72 With async programming, your program must never use blocking instructions and execution proceeds on a repeated cycle. PLCs solve this by not having any instructions which block, and many PLCs do not allow backwards jumps. Tradesmen in coveralls in factories were doing async programming decades before the cool kids heard about it. 73 Most PLC Programming Languages are Visual or Graphical Languages If ladder, Sequential Function Chart, or Grafcet and Function Block Diagram sound like visual programming languages, they are. In fact ladder is probably one of the earliest visual languages in commercial use. Wikipedia defines a visual programming language as: 74 In computing, a visual programming language (visual programming system, VPL, or, VPS), also known as diagrammatic programming, graphical programming or block coding, is a programming language that lets users create programs by manipulating program elements graphically rather than by specifying them textually. A VPL allows programming with visual expressions, spatial arrangements of text and graphic symbols, used either as elements of syntax or secondary notation. For example, many VPLs are based on the idea of "boxes and arrows", where boxes or other screen objects are treated as entities, connected by arrows, lines or arcs which represent relations. VPLs are generally the basis of low-code development platforms. Scratch is an example of a VPL 75 End of quote. As you can see, what is cool today was on the factory floor more than 40 years ago. -------------------- 76 Popularity of PLC Programming Languages PLCs are about as proprietary as you can get. Pretty much every vendor has his own proprietary take on each language. How you can program any particular PLC is determined by that vendor's programming software. If the vendor doesn't support it, you can't use it. An individual vendor may even have multiple incompatible product lines which have to be programmed in different ways with different software, although that is not as common these days as it once was. 77 Nearly all PLCs support programming in Ladder. Some support programming in instruction list as well as ladder. Anything else is much less common. 78 Ladder logic happens to be on the Tiobe Index by the way. For those who have not heard of it, the Tiobe Index, that is T I O B E, is a web site that lists the popularity of numerous programming languages based on various criteria. 79 Number one on their list happens to be Python, currently at 19.98%. Number two is C, at 11.55%. 80 At the time of writing this script, Ladder Logic was at number 46 with a 0.28% rating, which put it just below Erlang and just above Haskell. I'm not sure whether that means that Ladder Logic is not as obscure as you thought it was, or whether it means that Erlang and Haskell are in fact more obscure than you thought they were. -------------------- 81 Episode Summary In this episode we covered we covered The early history of computers in industrial control The early history of PLCs, including how they got their name Who the major brands are What they look like physically 82 A basic description of the abstract machine architecture A very brief look at what a PLC program is like The scan concept The main PLC programming languages The minor PLC programming languages The relative popularity of each of the programming languages 83 In the next episode we will take a look at one of the early PLCs from the era when they began seeing widespread use. This PLC was hugely successful and was for many companies the first PLC they used. This is the Allen Bradley PLC2. 84 This has been the second episode in an 8 part series. -------------------- Provide feedback on this episode.
-
-1
HPR4717: Visit with a blind Ham operator
This show has been flagged as Clean by the host. Hi, this is Archer72 for another episode of Hacker Public Radio. In this episode, it is my first experience troubleshooting with an experienced blind Ham operator who is located 20 miles away from my hometown. When traveling there, I could not miss his house, which was the only one off the main road with an estimated 50 foot antenna. That’s 15 meters for the rest of the world. This antenna was intended for the HF high frequency band, for which the local Ham radio club had very recently extended to this height. The target radio bands are 20, 40 and 80 meters. I had missed a great opportunity to record the session so wanted record (and write this) while it was fresh on my mind. Several months later… This particualar operator (KG4QWH) (Kilo-Golf-Four-Quebec-Whiskey-Hotel) was a radio operator for the local emergency services for 22 years, after which the new company decided to cut back on staff. A local broadcast radio station was hiring soon after and was using some version of Linux which was not accessible for a blind user like himself. This conversation was started after I was asked about his computer that he was trying to force update on older hardware from Windows (7?) to Windows 11, in true hacker fashion. While talking about the Linux situation and also the Windows side, it brough to light the glaring (near) ommision of accessibility software in both camps. On the Windows side, which is more familiar to my friend, is Jaws. Great software, but year after year the barrier of cost is getting more, and on the Linux side accessibility is not looking any better. While cost is not a factor in using it, and this is only coming from me, the voices could be better. Piper is an option for natural sounding voices, but has yet to be integrated into a FOSS screen reader AFAIK. On the positive side of accessibility, the Kenwood radio (model TS-590S) has the option to add a module to give an audible description of the buttons being pressed, and at the time there was gnother operator who came through and talk him through some receiver settings. The reason I went there is the first place was that the Kenwood microphone was causing noise to come across the speaker when he keyed up the mic. Initially the proprosed solution was to switch two wires inside the Kenwood microphone. I was asked about this ahead of time so was prepared with an electrical box with miscellaneous low voltage wires and stripping tool, also a Pincil soldering iron equiped with a USB C 100 watt power supply. To fix the problem on the Kenwood MC-60 microphone, you need to swap the wires so that mic hi is on Pin 1 and mic low (shield) is on Pin 7. This is often necessary due to factory wiring errors. After finding out that the proposed solution was to open the mic, which required a heat gun the soften the locktight, I tried another route. My friend had a toroid choke handy, and I coiled the feed line to the back of the radio around the toroid. (8 - 10 times?) This had no effect. Next to try was to ground the mic from the outside, which there was a bare metal screw on. This connected to a dedicated ground on the back of the Kenwood TS-590S radio. This seem to suppress the hum when using PTT. The rest of the evening (the best time for HF), Matthew was able to make contacts on 40 meters. Great to meet you as well my friend and I appreciate you coming up. I was able to talk on 40 this evening with no problems. I reached several in Kentucky Indiana and I even reached Kansas. Come to think of it. I was even able to reach someone on the Jersey shore. References Blind Hams Network New CQ Blind Hams Podcast: CQBH 011 TS-590SG Tutorial by N3AIN New CQ Blind Hams Podcast: CQBH 012 TS-590SG tutorial by N3AIN KENWOOD TS-590S instruction manual KENWOOD VGS-1 instruction manual KENWOOD Communications Microphone MC-60A Instruction Manual Image of a 15 meter HF antenna on a two story house in Kentucky. Provide feedback on this episode.
-
-2
HPR4716: PocketMod
This show has been flagged as Clean by the host. Last week, I discussed the "Second-Hand phone" approach to digital minimalism. In that episode I mentioned a calendar-making site "Pocket-Mod" Here is a review. the site: <a href="https://www.pocketmod.com">https://www.pocketmod.com.Provide feedback on this episode.
-
-3
HPR4715: friedcamp interviews part 2: Thomas Farstrike
This show has been flagged as Clean by the host. MicroPythonOS https://MicroPythonOS.org/ https://github.com/MicroPythonOS/MicroPythonOS Thomas Farstrike https://github.com/ThomasFarstrike Provide feedback on this episode.
-
-4
HPR4714: HPR Beer Garden 18 - Amber Ale
This show has been flagged as Clean by the host. The HPR Beer Garden returns for it's eighteenth episode with a look at Amber ales. Dave tries out Hobgoblin Amber , whilst Kevie samples Banks's Amber . Connect with the guys on Untappd : Dave Kevie The intro sounds for the show are used from: https://freesound.org/people/mixtus/sounds/329806/ https://freesound.org/people/j1987/sounds/123003/ https://freesound.org/people/greatsoundstube/sounds/628437/ The next three beer styles to be reviewed are: lager mild bitter Provide feedback on this episode.
-
-5
HPR4713: July2026 Community News response, and Future Plans
This show has been flagged as Explicit by the host. The show I'm responding to https://hackerpublicradio.org/eps/hpr4696/index.html The Album I plan on reviewing in the future (not the best album to start with getting into noise, but it's one of the more famous ones) https://open.spotify.com/album/2T2a5nzJ0uHJVhLkgiMeYA Reading on why I will never review anything by Boyd Rice (also if you don't mind paying for a patreon, the podcast The Empire Never Ended did a patreon exclusive episode on Boyd Rice and his ties to Neo-Nazi and Neo-Fascist groups) <a href="https://spencersunshine.com/2024/05/21/boyd-rice-neo-nazi-collaborator-video/">https://spencersunshine.com/2024/05/21/boyd-rice-neo-nazi-collaborator-video/Provide feedback on this episode.
-
-6
HPR4677: UNIX Curio #10 - Checksums and Hashes
This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. In UNIX Curio #8 ( HPR episode 4657 ), I talked about using standard utilities to compare files. Left unmentioned, however, was a method commonly used today—the hash function. As I've stated in previous entries, while I am an engineer, I don't have a background in computer science, so my understanding of the mathematics is limited. But I can give a practical description of what a hash function does. It takes an input, performs a set of calculations on it, and produces an output. As hash functions are practically used, the input is a set of bytes, such as a file or another piece of data like a password. The output is a numerical value in a fixed range—most often, expressed as hexadecimal characters. Because this "hash value" can always be represented in a certain number of bytes, its length as printed is usually a constant number of characters, padded with leading zeros if necessary. This episode will not cover the use of hashes in programming, focusing instead on using them to validate data. A hash function, or more specifically, a cryptographic hash function, has an additional property. It should be very difficult to predict what changes to the input would be required to produce a specific change in the output. An older, related concept is called a "checksum". While these are designed to vary when the input data is damaged or digits are transposed, they do not necessarily have that last property mentioned for cryptographic hashes. You have probably already encountered a checksum, even if you didn't recognize it. On a 16-digit number assigned to a Mastercard or Visa 1 credit or debit card, the first six digits identify the card issuer (such as a bank), the next nine digits are assigned to you by the issuer, and the last digit is a check digit. The check digit is calculated using the values of the previous 15 digits, and it is a simple way to avoid typos in entering a card number. In another example, every Ethernet frame that your devices send or receive includes a checksum 2 to help ensure that the contents weren't scrambled in transit. This is 32 bits long and is called a cyclical redundancy check, commonly referred to as a CRC. A CRC is also used in many other places—for example, the .zip file format includes one for each archive member, and this allows a program extracting files from the archive to identify if any were damaged. Our UNIX Curio for today is another example, the cksum utility 3 . It generates a 32-bit CRC based on the Ethernet algorithm. It operates on either a named file or standard input and outputs the CRC value, the length of the input, and the pathname if a file was given as an argument. Unlike most modern hashing programs, the checksum is printed as a decimal integer and is not padded, so it can be anywhere from one to ten digits long. The length value is the number of bytes in the input (actually specified as the number of octets , as systems could potentially use a byte that isn't eight bits long), also expressed as a decimal integer. There are two major ways that one could use cksum to check the validity of a file. First, if you are transferring a file from one UNIX-like system to another, you could run cksum against it on both systems and check that the CRC and length are the same. The utility can also be given multiple filenames as arguments, which would generate a list that can then be compared. The second way would be for someone publishing a file or set of files to also publish the CRC values, lengths, and names so that people downloading them could verify that they match. However, I don't think the practice of publishing lists like this really started until more recent hash functions like MD5 and SHA-1 came about so it is unlikely that anyone would publish CRC values instead. The advantage of these tools should be pretty obvious in comparison to cmp , one of the utilities discussed in UNIX Curio #8. To verify a file using cmp , you need two files to compare—if you're trying to check a large file you downloaded, you would need to spend the time and bandwidth to download a second copy. And if they didn't match, you would have no idea which of the two, if either, was correct. By contrast, cksum is quicker to run, doesn't require downloading a massive amount of excess data, and if run against the original file, makes clear what the correct value is. This utility is a follow-on from a program called sum , which operated very much the same. I had a bit of trouble tracking down the exact development history, but what seems clear is that two different variants 4 were popular: a BSD version and a System V version. Both output 16-bit checksums, but used different algorithms so they didn't give the same results. Also, the BSD version printed the length of the input data as the number of 1,024-byte blocks, while the System V version instead gave a count of 512-byte blocks. (Some sources claim that System V sum generates a 32-bit checksum 5 , which could possibly be true internal to the algorithm, but I have tested several independent implementations of the utility and all of them output a 16-bit value for both the System V and BSD algorithms.) From what I can tell, the BSD version 6,7 came first; it was in 3BSD but probably appeared even earlier. An identical copy of BSD's sum was included with UNIX/32V 8,9 , which was AT&T's 1979 port of Seventh Edition UNIX to the VAX and became one of the ancestors of System III. The divergence seems to have started with System III, released in 1980; its version of the sum utility 10,11 changed to a new default algorithm, though it could be made to use the BSD algorithm via the -r option. System V looks to have kept the same behavior as System III. It's not clear to me why this algorithm is universally called the "System V algorithm" rather than the "System III algorithm"; perhaps it is because System V saw much more widespread use. Instead of trying to reconcile these differences, the POSIX committee decided to create a new utility with a unique name, use a separate algorithm entirely, and avoid the block-length dispute by printing the length in octets instead of blocks. I should point out that POSIX states that the CRC algorithm for cksum does not strictly meet the mathematical definition of a "checksum". I don't know enough to say exactly why it doesn't qualify or to say whether either of the sum algorithms do. However, in less-formal usage the term "checksum" has gathered the meaning of any value used to represent or validate a set of data, so I am fine with using it no matter the technical details of the algorithm. When two different inputs produce the same checksum or hash value, this is called a "collision". Because the output value has a limited range, there are an infinite number of possible inputs that could produce a collision. From a practical standpoint the possibilities are more limited—the majority of these inputs are larger than the number of atoms in the universe, which can't fit on any machine. Unlike a cryptographic hash algorithm, the CRC is not specifically designed to resist an attacker crafting a malicious input that would cause a collision. However, it should be sufficient to detect accidental damage. Programs implementing more modern cryptographic hash algorithms are superior to the checksum utilities in avoiding collisions (whether malicious or accidental), but there are still three advantages that the older programs have. First, a system running a historical operating system might not have the hash programs available, but is more likely to have cksum or sum already included. Second, the checksum values are much shorter than the hashes output by the newer programs, making them easier for a user to compare by looking at them. This advantage is not as great as it might appear at first, because a common way to check a hash these days is to save a list of hashes and filenames—the hash programs can use that and do the comparison themselves, sparing the user from having to validate it character by character. The third advantage is that cksum prints the input length in bytes. This greatly limits the number of inputs that could be maliciously crafted to create a collision. I did a moderate amount of research on implementations of modern cryptographic hash algorithms and found that some, such as MD5, SHA-1, and SHA-2, do use the length of the input (often termed "message length" in the literature) as part of the material fed in to the algorithm, but none of the hashing utilities present this length to the user as part of its output. There are two possible reasons for this that seem evident to me. First, if one is hashing a password, you would certainly not want to give a clear indication of its length—that would give any attacker a massive head start on guessing the password. However, that doesn't explain why one would avoid printing the input length for a file that is made publicly available. Second, it is convenient in many contexts, such as database entries or in software (such as git ), for the hash to be a fixed length. Including an extra value that can be of variable length would complicate those use cases. However, the length value could simply be dropped and they would be no worse off than they are currently. Historically on UNIX, password hashing was treated differently from checksumming files— the crypt() function 12 was used for passwords while sum and later cksum were used to confirm a file's integrity. So even rather early on, these two use cases employed algorithms with different properties, but I haven't dived into the history deeply enough to know how intentional this was. My discussion in this episode focuses on the file use case, so understand that I'm largely avoiding the topic of password hashing. Digital signatures are yet another use case, one that I'm ignoring entirely. Every few years, some security researcher declares a particular hash algorithm to be "broken" and that everyone should move over to a new one, which generally has a longer hash. While the larger hash space certainly reduces the opportunity for collisions, this disrupts workflows, such as publishing information about software releases by e-mail, which still tends to observe a 78-character limit on each line 13 , making it harder to include a list of hashes with filenames next to them. This is in addition to the work of modifying software and scripts to use the new algorithm and managing how to treat past data. It seems to me that publishing the input length along with the hash would make it far more difficult to craft a malicious input that matches both, but I haven't found discussion of that during my investigation. (See the Appendix for a possible implementation.) Perhaps someone listening can record a response episode for HPR explaining that. References: Payment card number https://en.wikipedia.org/wiki/Payment_card_number Ethernet frame: Frame check sequence https://en.wikipedia.org/wiki/Ethernet_frame#Frame_check_sequence Cksum specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/cksum.html GNU coreutils manual: sum https://www.gnu.org/software/coreutils/manual/html_node/sum-invocation.html FreeBSD 15.0 sum manual page https://man.freebsd.org/cgi/man.cgi?query=sum&sektion=1&manpath=FreeBSD+15.0-RELEASE+and+Ports 3BSD sum manual page https://www.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/man/man1/sum.1 3BSD sum source https://www.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/src/cmd/sum.c UNIX/32V sum manual page https://www.tuhs.org/cgi-bin/utree.pl?file=32V/usr/man/man1/sum.1 UNIX/32V sum source https://www.tuhs.org/cgi-bin/utree.pl?file=32V/usr/src/cmd/sum.c System III sum manual page https://www.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/man/man1/sum.1 System III sum source https://www.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/cmd/sum.c Crypt specification https://pubs.opengroup.org/onlinepubs/009695399/functions/crypt.html RFC 2822: Internet Message Format: Line Length Limits https://datatracker.ietf.org/doc/html/rfc2822#section-2.1.1 OpenSSH 10.1 released https://lwn.net/ml/all/[email protected]/ Appendix The MD5 hash algorithm was (and still is) widely used, but many people characterize it as being "broken" and discourage its use. Let us imagine a variant of this, called MD5.L, where the normal MD5 hash is followed by a "." character and the input length expressed as a hexadecimal number. Take, for example, the e-mail message announcing the release of OpenSSH 10.1 14 . At the bottom, it includes an SHA-1 hash and an SHA-2 256-bit hash for the available gzipped tar files. That longer hash is encoded with Base64 because if it were given as a hexadecimal number, it would make the line longer than 78 bytes. The MD5.L hash of the file would be one character shorter than the SHA-1 hash, as shown below. (The extra length of the name makes them both consume the same number of characters. The hashes shown are for the "portable" version of OpenSSH.) Some people claim SHA-1 is also broken, seeking to have people use newer and longer hash functions. For an attacker to compromise MD5.L in this example, they would not only have to create a valid tar file compressed with gzip containing a malicious payload having the right MD5 hash, that file would have to be exactly 1,972,831 bytes long (the decimal equivalent of 1e1a5f). While there are still many possible inputs that could be tried (256 1972831 , to be exact*), this is far fewer than the infinite possibilities for plain MD5, SHA-1, or SHA-2. If for some reason it is super important to have a fixed hash length, let's imagine another variation called MD5+L. In this one, instead of L being the input length, it is the input length modulo one terabyte (2 40 bytes), which can be represented by 10 hexadecimal characters, left-padded with zeros. While this approach substantially increases the number of possible inputs an attacker could try, it is likely that an intended victim would notice that the file they downloaded is larger (or smaller) than expected by that much. The MD5+L hash is longer than a SHA-1 hash, but still shorter than a 256-bit SHA-2 hash. SHA1 (openssh-10.1p1.tar.gz) = 7fd17b99d1beffb47cd380d64079e920bb0bd91f SHA256 (openssh-10.1p1.tar.gz) = ufx6K4JXlGem8vQ+SoHI4d/aYU3bT5slWq/XAgu/B1g= MD5.L (openssh-10.1p1.tar.gz) = 80dd9bb00a86519934710d05903fdf07.1e1a5f MD5+L (openssh-10.1p1.tar.gz) = 80dd9bb00a86519934710d05903fdf07+00001e1a5f Of course, if MD5 is considered to be too weak even with the inclusion of the length, one could produce a ".L" or "+L" version of any hash function. However, longer hashes will end up running into the 78-character limit. *This is a number with 4.75 million digits that the bc utility on my laptop took almost 5 minutes to calculate. Provide feedback on this episode.
-
-7
HPR4676: HPR Community News for June 2026
This show has been flagged as Explicit by the host. New hosts Welcome to our new hosts: Lennart Benschop Last Month's Shows Id Day Date Title Host 4651 Mon 2026-06-01 HPR Community News for May 2026 HPR Volunteers 4652 Tue 2026-06-02 simon says Brian-in-Ohio 4653 Wed 2026-06-03 Starting the Habit of Reading Thaj Sara 4654 Thu 2026-06-04 What's in my component Box? MrX 4655 Fri 2026-06-05 Tips for Glasses norrist 4656 Mon 2026-06-08 My review of musicozy sleep/exercise bluetooth headband Swift110 4657 Tue 2026-06-09 UNIX Curio #8 - Comparing Files Vance 4658 Wed 2026-06-10 Audio Revisited Whiskeyjack 4659 Thu 2026-06-11 Command Line Fun - Recording a show Kevie 4660 Fri 2026-06-12 Robert A. Heinlein: The Future History, Part 1 Ahuka 4661 Mon 2026-06-15 Laptop Computer Woes, or How I Learned to Love My Tech Hoarding Claudio Miranda 4662 Tue 2026-06-16 “What Are the Answers I Need, To the Questions I Don't Know Enough to Ask?” Antoine 4663 Wed 2026-06-17 The hallway track at T-DOSE Ken Fallon 4664 Thu 2026-06-18 No Input Mixing Tutorial and How to Build a Drone Box TheDUDE 4665 Fri 2026-06-19 Pokémon GO Lee 4666 Mon 2026-06-22 How I got into tech Lennart Benschop 4667 Tue 2026-06-23 UNIX Curio #9 - printf Vance 4668 Wed 2026-06-24 Nuclear Power Technology Follow Up on Safety Whiskeyjack 4669 Thu 2026-06-25 HPR Beer Garden 14 - Super Strong Lager Kevie 4670 Fri 2026-06-26 Playing Civilization V, Part 13 Ahuka 4671 Mon 2026-06-29 Protocal AI operat0r 4672 Tue 2026-06-30 Hey Mum, I'm on Spotify ! Ken Fallon Comments this month Past shows hpr4181 (2024-08-12) "Downloading out of copyright movies" by Bob. Jan said: "subtitles" (2026-06-29 21:55:12) hpr4633 (2026-05-06) "Ham Radio Licence" by Lee. RJ said: "Very interesting as always" (2026-06-02 11:22:47) Lee said: "Aerials" (2026-06-02 16:44:45) hpr4644 (2026-05-21) "Response to comments on HPR4424: Newsboat..." by Archer72. Archer72 said: "Not quite a complete script" (2026-06-07 22:26:34) Whiskeyjack said: "Response to hpr4644" (2026-06-08 18:01:33) Archer72 said: "Response to WhiskeyJack" (2026-06-12 16:50:42) Whiskeyjack said: "Response to Archer72 on HPR4644" (2026-06-13 11:13:49) Archer72 said: "EyeD3" (2026-06-13 21:10:48) hpr4649 (2026-05-28) "What did I do at work today? Part 3 Section 2" by Lee. Ken Fallon said: "Love PHP" (2026-06-02 10:52:54) candycanearter07 said: "Re: Love PHP" (2026-06-02 17:24:01) hpr4650 (2026-05-29) "Playing Civilization V, Part 12" by Ahuka. Antoine said: "Puppetting and Happiness" (2026-05-29 14:21:14) Kevin O'Brien said: "That's the algorithm" (2026-06-01 21:13:58) Antoine said: "#2 Thanks!" (2026-06-04 23:25:42) This month's shows hpr4651 (2026-06-01) "HPR Community News for May 2026" by HPR Volunteers. candycanearter07 said: "busy weeknd" (2026-06-02 02:38:55) hpr4653 (2026-06-03) "Starting the Habit of Reading" by Thaj Sara. Antoine said: "Reading always has been a life-saver for me, this alone makes this show Very Nice" (2026-06-04 23:37:27) Kevie said: "Nice episode" (2026-06-10 08:11:43) hpr4654 (2026-06-04) "What's in my component Box?" by MrX. Kevie said: "Geeks are ahead of their time" (2026-06-10 08:14:12) hpr4655 (2026-06-05) "Tips for Glasses" by norrist. Trey said: "Avoid hand soap " (2026-06-05 12:11:33) Jim DeVore said: "Geeks are lazy, too" (2026-06-21 00:10:45) Operat0r said: "Glasses" (2026-06-26 18:53:44) hpr4656 (2026-06-08) "My review of musicozy sleep/exercise bluetooth headband" by Swift110. candycanearter07 said: "wishlist!" (2026-06-08 13:12:08) Kevie said: "Interesting" (2026-06-10 08:15:59) hpr4657 (2026-06-09) "UNIX Curio #8 - Comparing Files" by Vance. xmanmonk said: "Great Show (again)" (2026-06-09 23:31:58) candycanearter07 said: "comparisons" (2026-06-11 16:57:57) Whiskeyjack said: "HPR4657 - use of comm" (2026-06-13 11:47:53) Vance said: "Appreciate the comments" (2026-06-15 03:16:05) Whiskeyjack said: "Reply to Vance on awk in HPR4657" (2026-06-15 17:02:57) hpr4659 (2026-06-11) "Command Line Fun - Recording a show" by Kevie. candycanearter07 said: "ffmpeg concatenation" (2026-06-15 11:58:49) Whiskeyjack said: "Reply to candycanearter07 on HPR4659" (2026-06-15 16:53:16) hpr4661 (2026-06-15) "Laptop Computer Woes, or How I Learned to Love My Tech Hoarding" by Claudio Miranda. candycanearter07 said: "uses :)" (2026-06-16 15:04:20) xmanmonk said: "Great show!" (2026-06-17 04:37:48) ClaudioM said: "Thanks! (In reply to candycanearter & xmanmonk)" (2026-06-17 14:02:17) hpr4666 (2026-06-22) "How I got into tech" by Lennart Benschop. Windigo said: "Nice to meet you" (2026-06-26 02:24:48) candycanearter07 said: "hello!" (2026-06-27 15:04:02) hpr4667 (2026-06-23) "UNIX Curio #9 - printf" by Vance. candycanearter07 said: "learned something new :)" (2026-06-23 13:23:06) Whiskeyjack said: "HPR4667 - printf" (2026-06-24 03:54:20) xmanmonk said: "Another great show!" (2026-06-24 21:15:22) Mailing List discussions Policy decisions surrounding HPR are taken by the community as a whole. This discussion takes place on the Mailing List which is open to all HPR listeners and contributors. The discussions are open and available on the HPR server under Mailman. The threaded discussions this month can be found here: https://lists.hackerpublicradio.com/pipermail/hpr/2026-June/thread.html Events Calendar With the kind permission of LWN.net we are linking to The LWN.net Community Calendar. Quoting the site: This is the LWN.net community event calendar, where we track events of interest to people using and developing Linux and free software. Clicking on individual events will take you to the appropriate web page. Provide feedback on this episode.
-
-8
HPR4675: Yard Inflatables
This show has been flagged as Clean by the host. SUMMARY Presenter discusses modular inflatable systems and challenges with lawn ornaments. IDEAS Modular inflatable systems for improved durability. Use of zip ties and 3D-printed parts for connectivity. Challenges with air flow and electrical connections. Need for modular lighting systems. Repair techniques for inflatable seams. Importance of quality materials for longevity. Combining air and electrical systems in one tube. DIY solutions for extending inflatable life. Issues with cheap lawn ornaments and their components. Industrial blow fans for large inflatables. Protecting wires from weather damage. Replacing faulty motors in inflatables. Balancing air pressure to prevent leaks. Modular design for easy maintenance. Cost-effective solutions for outdoor decorations. Challenges with outdoor charging boxes. Enhancing durability through adhesive treatments. Need for seamless integration of components. Longevity of inflatables under continuous use. Practical approaches to inflatable system design. RECOMMENDATIONS Use modular designs for easier repairs. Incorporate 3D-printed connectors for durability. Protect electrical components from moisture. Opt for industrial-grade fans for large inflatables. Combine air and electrical systems in one tube. Apply adhesive treatments to prevent seam splitting. Replace faulty motors with compatible alternatives. Balance air pressure to avoid overfilling. Use high-quality materials for longer lifespan. Implement DIY repair solutions for cost-effectiveness. Design seamless connections for improved functionality. Prioritize weather-resistant components. Create modular lighting systems for flexibility. Test inflatables under continuous use conditions. Use spray adhesives to extend inflatable life. Simplify assembly for user-friendly installation. Explore cost-effective materials without compromising quality. Address common failure points in inflatable systems. Develop standardized repair techniques. Focus on integrated design for longevity. Provide feedback on this episode.
-
-9
HPR4674: Audiobooks
This show has been flagged as Clean by the host. Lee recorded a show 4511 which aired on November 17, 2025. Since I had some observations of my own I decided to record a show in response. I have enjoyed some significant series of audiobooks, both in Science Fiction and in History, as they are both topics I greatly enjoy. And many of these are available online. Links https://www.openculture.com/freeaudiobooks https://librivox.org https://archive.org/download/IsaacAsimovAudioBookCollection https://www.google.com/search?client=ubuntu-sn&channel=fs&q=robert+heinlein+audiobooks+youtube https://archive.org/details/arthur-c.-clarke-audiobooks-two https://librivox.org/author/808 https://archive.org/details/WillDurant-TheStoryOfCivilizationVolume1BookmarkableM4bFile Provide feedback on this episode.
-
-10
HPR4673: First contact conversation
This show has been flagged as Clean by the host. Hello, this is your host, Archer72 for another episode of Hacker Public radio. In this episode, I make my first contact off a local repeater in a small town in Kentucky. What got me to try my hand at radio? It was when I started to capture ISS (International Space Station) Ham radio transmissions. From Kentucky, I have logged receiving from Texas and Oregon and as far as a brief transmission over Great Britain. My next step was to have a first contact, which I botched at first when I said my call sign, and didn’t leave enough time for someone to respond. I thought that a repeater was the best place to start, so found this one on RepeaterBook.com The repeater was found in RepeaterBook at RepeaterBook : My local Cynthiana, Kentucky repeater Youtube : Harrison County Amateur Radio Club QRZ : Harrison County Amateur Radio Club Facebook : Harrison County Amateur Radio Club The callsign for this repeater is Kilo-Four-Kilo-Juliett-Quebec It is currently On Air (and quite active near club time and the weekend) The base frequency is 147.165 MHz with an offset of plus (+) 600 KHz and a tone to open the repeater, of 67.0 Hz It is hosted by the Bluegrass Amateur Radio Society The RepeaterBook entry was updated on 2025-10-27 K4KJQ On-Air Open 147.16500 (+) MHz 67.0 Cynthiana • Harrison County • Kentucky Updated 2025-10-27 Technical Downlink 147.16500 Uplink 147.76500 Offset +0.600 Uplink Tone 67.0 Downlink Tone 67.0 Antenna (AGL) 125 feet Sponsor: Bluegrass Amateur Radio Society Sponsor: BARS Local Time 04:30 (EDT UTC-04:00 DST) America/New_York Reviewed 2025-10-27 Bluegrass Amateur Radio Society RepeaterBook Worldwide RepeaterBook is a worldwide amateur radio repeater directory So, with after all that being said I programmed a Baofeng BF-F8HP to this frequency manually. You can also use open source Chirp software, but I wanted to know my radio to program it on the fly. This brings me to a recording with permission from Keith (KO4BWJ) who is located outside this little A notable feature of this the town of Cynthiana, Kentucky is the hometown of the artist Robert Kirkman for the Television series The Walking Dead, and home to the mural depicting the main characters of the show. This mural had some controversy over the rights to have it depicted, or so I thought. The actual story is story is that some groups did not want a Zombie theme representing their town. Cynthiana, Kentucky: Walking Dead Mural Without further delay, here is the promised recording. … I hope you enjoyed and learned from this little piece of an experience Ham vs a new (to the airwaves) Ham. This has been Archer72 (Kilo-Delta-Niner-Victor-Mike-Whiskey) 73 Provide feedback on this episode.
-
-11
HPR4672: Hey Mum, I'm on Spotify !
This show has been flagged as Clean by the host. The Technical Dutch Open Source Event (T-DOSE) is a free conference to promote the use and development of Open Source software. This event has is organised yearly since 2006 in the Brainport region, near Eindhoven, The Netherlands. During this event, Open Source projects, developers and visitors can exchange ideas and knowledge. Ken gave a presentation, you can watch the recording https://www.youtube.com/watch?v=6gmX-Ap7knI&t=10912s Provide feedback on this episode.
-
-12
HPR4671: Protocal AI
This show has been flagged as Explicit by the host. In this episode, Operator dives into his ongoing journey to migrate away from centralized cloud ecosystems specifically moving his daily workflow off Google Keep and onto Obsidian hosted locally on a Debian server. Operating purely over a secure VPN to minimize his external attack surface, he discusses the security considerations of managing personal data in local plain-text markdown files. The episode features a deep dive into local AI infrastructure, sparked by technologist Daniel Miessler’s recent shift away from RAG (Retrieval-Augmented Generation) in favor of a simpler, localized file-system-as-context approach (using fast search tools like ripgrep). Operator shares his own mixed results experimenting with RAG noting great success with massive, structured car repair manuals, but incredibly poor fidelity when indexing conversational podcast transcripts. To find the sweet spot, Operator is testing a dual approach : combining flat-file local search with a PostgreSQL vector database ( pgvector ). He also rants about the frustrating "hype cycle" of online tutorials that claim to teach "local" setups but secretly rely on expensive, cloud-hosted frontier models. Finally, the host introduces his ambitious roadmap for "Protocol AI." Designed as a localized, read-only dashboard to help manage his ADHD and "time blindness," this system will scrape, aggregate, and summarize his cluttered digital life including multiple Gmail accounts, Yahoo spam, calendars, and a massive array of social media feeds (Signal, Discord, Mastodon, BlueSky). The long-term goal? Transitioning from a read-only local summarizer to a safe, "human-in-the-loop" execution assistant that keeps his data out of the hands of mega-corporations. References Obsidian is a proprietary personal knowledge base and note-taking application that operates on Markdown files. The software is free for personal and commercial use; only the offered cloud services, optional commercial licenses, and early access versions are paid. It is available as desktop versions for macOS, Windows and Linux as well as for mobile operating systems such as iOS and Android, but not as a web application. Obsidian - From Wikipedia, the free encyclopedia Retrieval-augmented generation (RAG) is a technique that enables large language models (LLMs) to retrieve and incorporate new information from external data sources. With RAG, LLMs first refer to a specified set of documents, then respond to user queries. These documents supplement information from the LLM's pre-existing training data. This allows LLMs to use domain-specific and/or updated information that is not available in the training data. For example, this enables LLM-based chatbots to access internal company data or generate responses based on authoritative sources. RAG (Retrieval-Augmented Generation)Provide feedback on this episode.
-
-13
HPR4670: Playing Civilization V, Part 13
This show has been flagged as Clean by the host. In our sample game we look at playing as Austria and aiming for a Diplomatic Victory. And our focus is on puppeting City-States, but I misunderstood and instead of making a Diplomatic Victory easier, it makes it harder. I still managed to get my Diplomatic Victory, but a Science or Domination Victory would definitely have been easier in this scenario. Playing Civilization V, Part 13 Playing for a Diplomatic Victory, Part 2 So, how do you solve a Happiness problem? For a full discussion, you can check out this section of the Civilization Wiki. In Civilization V you have these options: Trade for luxuries – Luxury resources are the best way to get a quick hit of Happiness, and trade is the best way to do this quickly. But to trade you have to have met the other civ, so my drive to explore pays off here. Every luxury resource I can obtain adds +4 Global Happiness to my Empire. Note that only the first unit of the resources adds happiness. So if you have multiple copies of a luxury resource you should keep one and trade the others if you can. Buildings – Buildings like the Circus (+2), Colosseum (+2), Zoo (+2), and Stadium (+4) provide local happiness in the city that builds them. That still gets added to your total happiness number, but local happiness can never exceed the size of the population. So if you have a city with 5 population, building a Circus and a Colosseum might make sense, but building a Circus and a Stadium would not since 2 of the Happiness would not count. Buildings take longer than Trade deals, so if you need a quick hit start with trading for Luxuries. Wonders – Chichen Itza (+4), Notre Dame (+10), Taj Mahal (+4), Eiffel Tower (+5), Neuschwanstein (+1 for every Castle you have), Prora (+1, plus +1 for every 2 Social Policies you have adopted), and CN Tower (+1 per city). Wonders are even harder to build, and you may not get them since you are competing with other players. But there is a National Wonder worth getting, the Circus Maximus (+5) Social Policies – Every policy tree has policies that can add to Happiness or decrease Unhappiness, which is just as good. Natural Wonders – Each Natural Wonder grants +1 Global Happiness to each empire that has discovered it. So again exploration is key. Mercantile City States grant +3 Global Happiness to each Empire that is at least Friends with it. So you have a lot of options here, and they are listed in order of how you should pursue them. Trading for Luxury Resources can be done quickly and provide an immediate boost, so you should be actively trading all along. Buildings and Wonders should definitely be pursued, but here you will probably be balancing demands for other investments as well. Natural Wonders are nice, but you need to explore the whole map to max that out. I was able to trade one of my Whales to Patchacuti for one of his Ivory, which helped push my happiness up. Then with Askia I traded one Spice for one Copper. I also made a small gift to the City-State of Manila to make them my ally. Once 5 turns have elapsed, I can take them over via Diplomatic Marriage. And I have started on Colosseums in several of my cities to push up my Happiness further. Meanwhile I continued my exporations, and became the first to discover all of the the other Empires in the game. This makes me the host for the World Council, which is good for one additional vote. I will keep exploring since there are probably some Natural Wonders out there worth finding yet. And indeed on the next turn I found Mount Fuji, pushing my Happiness to 24. Then I Puppeted Manilaa, and over a couple of turns it dropped to 18. But a Colosseum got built, and I found a Natural Wonder,so I am back to 21. I am not the leader in this game, and I think I can afford to Puppet another City-State. For now I am avoiding doing this to Mercantile City-States since they can add to my Happiness. But there’s lots of City-States in this game. I picked Kathmandu for my next target, but also dropped 250 on Ragusa to become Friends with them. You have to look to the future sometimes. I am currently pulling in 144 Gold per turn, so I can afford it. At turn 267 my Happiness was up to 30, but as soon as I puppeted Kathmandu it dropped to 22. This is why you need to manage your Happiness carefully. This pushed me another 100 points up on the Leader board, so I am in good shape. But with 22 Happiness, I can probably afford to start on puppeting Ragusa. At turn 272 I have 5766 Gold, and I am earninig 163 Gold per turn. My Happiness is at 27, so I am in good shape. I puppet Ragusa, and my Happiness falls to 18, but my status on the leader board goes up another 100 points, putting me 200 ahead of my nearest competitor. But a few turns later and my Happiness is back up to 24, so I am starting the process to puppet Bucharest. At turn 280 I have 6584 Gold, I am earning 179 per turn, and my Happiness is 26. Then I puppet Bucharest, and my Happiness drops to 16. But I am nearly 300 ahead of my nearest competitor on the leader board, so not bad. Also, we are getting near to the first World Council, where I am the host. I have put a proposal on the agenda to enact a World’s Fair. The number 2 player has put a proposal on the ban Whales as a luxury resource, and I have a lot of whale resources. I suspect the World’s Fair will pass without my votes since many other players supported it, so I will put my votes into denying the Whale ban. We’ll see how it goes. I have the most votes, but if everyone else gangs up on me they can get it passed. And as I expected, the World’s Fair passed without my votes, and the Whale ban was defeated. I tabled a proposal to enact Arts Funding for the next Council. It looks to be very popular. Right now I starting to snowball. On turn 287 I now have 6476 Gold, and I am earning 231 each turn. I have puppeted 6 City-States, and my Happiness is at 31. I can now start to puppet another. In the World Council I have 4 votes, and every other player has 1. My main strategy right now is to keep on friendly terms with the other players since everything is going well. But I am also starting to build factories to improve my Production capacity, which in turn wil let me update my Military. I picked up a Great Scientist, and used him to construct an Academy to boost my Science output. And every city next to a mountain has gotten an Observatory for added science . And then I constructed the Leaning Tower of Pisa and got one free Great Person of my choice, and I chose a Great Scientist and constructed another academy. And I hit a Golden Age good for 10 turns. My Golden Age ended, but on the next turn I finished the Taj Mahal which gave me another 10 turn Golden Age, in addition to =4 Global Happiness. And I used some of my big cash balance to upgrade my military units, because in this game the best way to keep the peace is to have a strong military. Then on the next turn I was able to choose an ideology, so I picked Freedom, and for my two policies I chose Universal Healthcare and Capitalism, each of which added Happiness. I have earned a couple of Great Artists, and I plan to use them to create more Golden Ages since accumulating Culture is not part of my strategy. On turn 309 I puppeted Bratislava and started on Wittenberg. And now that I have expxlored the entire map I don’t need my Caravels any longer, so I deleted them. then on turn 314 my closest competitor, Japan, denounced me, and was in turn denounced by Brazil. I have been building up my military, so I am not worried. Basically, Japan has realized I am opening up a bigger lead, and that is why they denounced me. I’d like to win without going to war, but if he attacks me I can handle it. Meanwhile, I am also building up my Science to open up a lead there. On the next turn I built the Red Fort, which improves my defensive strength. Brazil approached me to join in a war on Japan, but I declined. I am not seeking a military victory in this game, so I don’t want to be distracted. However I just puppeted two more cities, and looking at the map I now have 3 cities on the border of Japan. So maybe there will be a war in my future. I got another policy to choose, and picked Universal Suffrage, which increases the length of Golden Ages, since I plan a few. And I am starting to build more military units, starting with riflemen which are good for defending cities. If I am given time I will ship them to my puppeted cities near Japan. But if he attackes me before then, they are still worth having. Links https://civilization.fandom.com/wiki/Happiness_(Civ5) https://www.palain.com/gaming/civilization-v/playing-civilization-v-part-13/ Provide feedback on this episode.
-
-14
HPR4669: HPR Beer Garden 14 - Super Strong Lager
This show has been flagged as Clean by the host. In the 14th episode of the HPR Beer Garden, Dave and Kevie are joined by regular guest Paul to sample super strong lager. Known in the media as 'Tramp Juice', the guys look at whether this infamous style is worth a try and how it rates as a beer when you look past the stigma. Dave samples Skol Super, Kevie tries Kestrel Super and Paul opts for Petra Mocna from Poland. Upcoming beers: Double IPA Belgian Blonde Double Dry Hopped (DDH) IPA Connect with the guys on Untappd : Dave Paul Kevie The intro sounds for the show are used from: https://freesound.org/people/mixtus/sounds/329806/ https://freesound.org/people/j1987/sounds/123003/ https://freesound.org/people/greatsoundstube/sounds/628437/ Provide feedback on this episode.
-
-15
HPR4668: Nuclear Power Technology Follow Up on Safety
This show has been flagged as Clean by the host. -------------------- 01 Introduction This is the second follow up to my 8 part series on nuclear power. In this episode I will attempt to answer a question posed by brian in ohio in a comment on HPR4583. In that comment he said: 02 -------------------- Loving this series. Maybe Whiskey Jack could give some cost comparisons between large and small reactors. He could also give us a realistic look at nuclear plant safety/accidents compared to conventional power production. Looking forward to the episode on FORTH generation reactors ;-) -------------------- 03 End of quote. The first question I answered in my previous follow up, which was HPR4628. In this episode I will attempt to answer the second question, which was about the safety of nuclear power compared to other sources of electrical power generation. One of the HPR janitors encouraged me to make this episode, so I think we can thank him for getting another HPR episode made. 04 Defining the Scope First, let's define the scope of the question. This will cover electrical power generation only. Within that scope I will consider only the following sources of energy. 05 Coal Oil Natural Gas Hydroelectric Nuclear Wind Solar I won't cover geothermal, wave, or tidal power as these are only used in very small amounts and so there simply isn't enough literature on them to base a discussion on . 06 Foreshadow Conclusion I should mention right away that I cannot provide absolute answers to this question in the form of a nice, neat ranking table based on numbers from peer reviewed scientific sources. The reasons for this will become apparent, but to put it briefly, the data on which to base such a ranking simply doesn't exist. I will however provide context within which people can think about the issue. Wherever possible, I will provide links to the references that I used in the show notes so you can read further on this yourself. -------------------- 07 Energy Catastrophism versus Energy Uniformitarianism First though I need to go off on a slight geological detour in order to explain an important analogy that I will use. 08 In the 19th century there was a great debate among geologists over what is known as catastrophism versus uniformitarianism. In seeking to explain the origins of the earth and of the landscape that we see around us, there were two points of view. 09 One was "catastrophism". This is the belief that the mountains, valleys, and plains that we see around us were formed as a result of great catastrophes which occurred relatively recently in earth's history. This explanation was necessary in order to fit geological features into an earth that was believed to be only a few thousands of years old. This view was heavily influenced by religious belief. In this view Noah's flood was the great catastrophe and the fossils of dinosaurs were the remains of animals who had not been saved on the ark and so had died in the flood. 10 The other point of view was uniformitarianism. This was the hypothesis that the landscape we see around us can be explained by the very slow accumulation of very small changes over very long periods of time. For this to be true however, the earth had to be far older than the few thousand years that a literal reading of the bible would suggest. The earth in fact had to be many, many, millions of years old. 11 Eventually, the uniformitarian view won out and people understood that while some catastrophes can take place, the shape of the landscape is overwhelmingly due to small changes over very long periods of time. 12 How is this Relevant to this Episode You Ask? How this is relevant is that I will use this analogy to explain how we need to think about energy and safety. Very small numbers of deaths and injuries multiplied over many occurrences can add up to big numbers, comparable in scale or possibly even larger than a single catastrophe or even several of them. 13 I don't know if anyone else has used this analogy before, I have just thought of this when writing the script for this podcast. None the less, I think it is a very useful way of helping to understand the issues. 14 As an example of this, think about the well known case of the safety of flying versus the safety of travelling in your car. Air crashes are catastrophes that make the headlines. Automobile crashes are seldom more than local news at best. You have probably heard many times the claim that if you making a trip somewhere, you are safer to fly than to drive yourself in your car. 15 Example - Hydro versus Solar I will now present an example of this. Hydro electric power has some notable large scale catastrophes associated with it. Roof top solar power does not have any notable catastrophes that I am aware of. However, which is safer? 16 Hydro Catastrophes Here are three examples of hydro electric catastrophes in just one country, Italy. The Vajont Dam which collapsed in1963 An estimated 1,917 to 2,500 people died. The Sella Zerbino dam which collapsed in 1935. More than 100 people died. The Gleno Dam which collapsed in 1923. An estimated 350 people died. https://damfailures.org/ https://pmc.ncbi.nlm.nih.gov/articles/PMC4997708/ 17 I haven't tried to compile a global list of the worst hydro electric dam collapses, as this sort of information is actually very difficult to find, even on web sites dedicated to dam failures. An additional problem is that information on whether a dam was used for electric power generation or not is often not available. 18 Dam failures where contradictory or insufficient information is available on whether there was an associated hydro power plant include the 1975 Banqian Dam failure, where death estimates range up to a quarter of a million. 19 Solar Panel Slow Accumulation Contrast this with roof top solar panels. Many small accidents can add up to big numbers as well. 20 Health and safety literature discussing solar panel safety mention things such as Falls from roofs. Electric shock. Arc flash (burns from electrical arcing). Normal electrical safety procedures which are based around locking out sources of energy do not work with solar panels which makes safety more difficult. Heat stress due to working exposed in the hot sun. Warning from US government on falls by solar panel installers. https://stacks.cdc.gov/view/cdc/228946 https://www.osha.gov/green-jobs/solar 21 Why We Cannot Compare the Two Hydro catastrophes are not well documented, but we can at least find records of some of the most notable ones. However, even those have very large variations in estimates of deaths. 22 Roof top solar deaths however are largely undocumented. The industry is largely unregulated. There is no central authority which accumulates many individual deaths or injuries. At best there are worker and public safety bodies who simply accumulate those statistics into general construction or household injuries. 23 Thus we have no reliable means of comparing the two energy sources on a comparable basis. We face the same problem with all other major electrical energy sources. So far as I am aware, there are no peer reviewed scientific studies which compare the relative safety of all of the major electrical energy sources we are considering here based on actual numbers. -------------------- 24 Safety Risks I will now try to list some the major hazards for each of energy sources we are considering. There is however limited data available. In many cases we just have reference to worker safety organizations as to what the hazards are. I will not attempt here to put numbers to these here. Categories 25 Coal, Oil, Natural Gas The hazards are Air pollution Mining and oil field accidents Pipeline explosions Transportation accidents. These- move a lot of material so these are significant. 26 Hydroelectric These include Dam collapse Drowning 27 Nuclear These include Radiation exposure 28 Wind These include Falls Confined space deaths (there is not much detail on this) Electric shock Ice throws (that is, throwing pieces of ice off the blades) This technology has a significant problem with people working alone which greatly increases risks associated with other dangers. 29 Solar These include Falls Electric shock Arc flash Heat stress 30 I have not tried to cover all possible risks associated with each category, just the ones which each industry considers to be the risks they concern themselves with. There does not exist any means by which risks of similar types are compared across different industries. 31 Reliability of Supply is Also Safety In a completely electrified net zero society, reliability of supply is a safety matter. People will die in very large numbers in cold climates if they do not have heat. If we have no fossil fuels, we need to also consider how reliably does a grid based on any of the options work. I have not seen anyone attempt to address this question and will not attempt to address it here. However, it must be addressed in any comprehensive attempt to rank safety. -------------------- 32 Studies or Articles on Estimates of Relative Safety Despite the difficulties of comparing the safety of different sources of energy, some people have attempted this anyway. Different estimates done at different times had different focuses, so unfortunately we do not have a nice set of studies that we can neatly use to cross check one another. I will however list the names and the authors and summarize the results. -------------------- 33 The Health Hazards of Not Going Nuclear By Dr. Petr Beckman Published in 1976 The author of this book tried to address the relative safety of different sources of energy in the mid 1970s. However, it is old at this point, so I won't bother digging through its pages to find his figures. 34 He mainly focused on comparing electric power generated with coal to nuclear. His conclusion was that if the goal was to prevent deaths or ill health in the process of generating electricity, then the logical conclusion was to replace coal fired power plants with nuclear. 35 The book was relatively well known at the time, as least as far as books on energy are concerned, so I thought it was still worth mentioning. I happen to have a copy of this book which I bought back in that time period It was the 8th printing of the book, so it would appear to have had relatively good sales. 36 The author did address the issue of what I have termed "catastrophism" in his comparison of different energy sources, although I don't know if he used this phrase. I don't know if he was the first to use this sort of analysis, but he certainly was very influential in terms of popularizing it. -------------------- 37 Risk of Energy Production by Herbert Inhaber Publication AECB 1119 March 1978 This study is a scientific paper from the same time period as the book "The Health Hazards of Not Going Nuclear". 38 He based his risk estimates largely on estimates of the amount of material which was used in the construction and operation of various power sources. While we could argue over whether or not this is a valid methodology, I think any such argument would be pointless as I think the age of the study alone renders it not relevant today anyway. Advancements in materials have changed the basis results significantly by now. However, as it exists I thought I would mention it to show that the idea of comparing energy sources to each other is not a new one. The author compared a wider variety of potential sources than Beckman did. 39 Here's his conclusions. He assumes equal amounts of energy produced by each method. The numbers are normalized such that the total sums to 100%. You can think of it in terms of what proportion of total deaths or injuries would result from each source if each were equally used. 40 Coal 27.5% Oil 25.6% Methanol 16.7% Wind 10.8% Solar photovoltaic 9.2% Thermal 8.1% Solar space heating 1.5% Ocean thermal 0.4% Nuclear 0.13% Natural Gas 0.08% 41 His natural gas estimate is drastically different from that of other authors. I am not going to worry about explaining it however, as the study is as I said old enough to be not very relevant anyway. I am mainly including this here out of historical interest. 42 As a footnote, the methanol he refers to would be synthesized from wood. This was a popular idea in that era as a means of providing liquid fuels for transportation. Practical battery electric cars in those days were strictly science fiction. 43 The ocean thermal category is a real blast from the past and I had forgotten all about that concept. It was a very popular idea at that time and was supposed to be *the* big and upcoming thing in renewable energy. It involved various means of attempting to extract energy from differences in water temperature at different depths in the ocean. It gradually faded away however, as despite great efforts being put into it, designs never proved to be practical. -------------------- 44 Electricity generation and health Anil Markandya, Paul Wilkinson Published in the Lancet, Vol 370, 15 September 2007 45 This is more recent than the previous one, although it is nearly 20 years old at this point. Unfortunately it doesn't cover wind or solar, just fossil fuels and nuclear. However it is still useful, and the Lancet is a very reputable peer reviewed journal. 46 I will present just the results rather than discussing the whole paper. The authors break it down into deaths among the public, occupational deaths, and air pollution related deaths, serious illness, and minor illness. 47 They break the energy sources down into lignite, coal, gas, oil, biomass, and nuclear. Lignite is a type of very low grade coal used mainly for electric power generation. In this paper biomass refers to energy crops and forest residues. 48 I will summarize the results by category rather than trying to describe a table that has 6 rows and 5 columns. All numbers are normalized in terms of deaths or cases per TWh. 49 Occupational deaths from accidents lignite 0.1 coal 0.1 gas 0.001 oil no data biomass - no data Nuclear is 0.019. 50 Deaths among the public from accidents lignite 0.02 coal 0.02 gas 0.02 oil 0.03 biomass no data Nuclear 0.003 51 Air pollution deaths lignite 32.6 coal 24.5 gas 2.8 oil 18.4 biomass 4.63 Nuclear 0.052 52 Air pollution serious illnesses lignite 298 coal 225 gas 30 oil 161 biomass 43 Nuclear 0.22 53 Air pollution minor illnesses lignite 17,676 coal 13,288 gas 703 oil 9,551 biomass 2,276 Nuclear no data 54 Natural gas edges out nuclear power slightly in terms of occupational safety, but in every other category nuclear is drastically lower in terms of ill effects than any of the alternatives. -------------------- 55 2020 Fatalities for US Roofers Increased 15% as Solar Roof Installations Increase Published in The Next Big Future July 6, 2021 by Brian Wang 56 This seems to be written by someone who has a popular science blog. I'm not familiar with it personally, but he addresses the subject so I'll list it. The title implies that it's all about rooftop solar, but he provides comparative numbers for the other energy sources of interest, so that is useful for our purposes. However, he doesn't describe his methodology, so we need to treat them with some caution. Here are his results These are deaths per thousand terawatt hours. 57 Coal - 100,000 Oil - 36,000 Natural gas - 4,000 Hydro - 1,400 Rooftop solar - 440 Wind - 150 Nuclear - 90 58 If we plot these numbers on a bar chart, coal and oil are so large that all of the others are squished to the bottom of the chart and are difficult to see at all. Let's therefore look at these in terms of orders of magnitude. Keep in mind that this is a logarithmic scale. This means that the difference between 4 and 5 is much greater in linear terms than the difference between 1 and 2. 59 Coal - 5 Oil - 4 Natural gas - 3 Hydro - 3 Rooftop solar - 2 Wind - 2 Nuclear - 1 60 Each of these numbers represents an order of magnitude, that is a power of ten. We can see that with rooftop solar, wind, and nuclear, the numbers are so close and the uncertainties are so great and their relative values so small compared to say coal that they can be seen as equivalent so far as safety is concerned. -------------------- 61 What are the safest and cleanest sources of energy? by Hannah Ritchie Published in Our World in Data First published in 2017, updated in 2022 and 2024 62 The author of this study addressed both deaths and greenhouse gas emissions. Deaths from accidents and air pollution are normalized to per TWh of electricity, while greenhouse gas emissions are normalized to GWh of electricity over the life cycle of the plant. 63 Here are the death figures. Coal 24.6 Oil 18.4 Biomass 4.6 Natural Gas 2.8 Hydro power 1.3 Wind 0.04 Nuclear 0.03 Solar 0.02 64 For greenhouse gas emissions the figures are Coal 970 tons Oil 720 tons Natural gas 440 tons Biomass 78 to 230 tons Solar 53 tons Hydro power 24 tons Wind 11 tons Nuclear 6 tons 65 If we take the death figures and rank them by order of magnitude as we did with the previous article, we get the following. 66 Coal - 4 Oil - 4 Biomass - 3 Natural Gas - 3 Hydro power - 3 Wind - 1 Nuclear - 1 Solar - 1 67 Keep in mind that the previous article covered only rooftop solar and not large industrial installations, and so is not directly comparable. Also the units are different, with the previous article being in terms of thousand TWh, and this one being in TWh. If we exclude solar (as the numbers are not comparable), Brian Wang's numbers are between 1.5 to 4 times higher than Ritchie's, except for hydro which are almost identical. I think this latter is due to both sets of numbers are dominated by one exceptionally big hydro accident. 68 Overall however, the relative rankings are quite comparable. Ritchie's numbers for deaths from coal, oil, and natural gas appear to be directly from the study by Markandya and Wilkinson mentioned above. For the benefit of those who are wondering, Ritchie specifically states that her numbers for nuclear include the Chernobyl and Fukushima accidents. -------------------- https://www.iaea.org/publications/magazines/bulletin/21-1/solar-power-more-dangerous-nuclear Direct link to file https://www.iaea.org/sites/default/files/publications/magazines/bulletin/bull21-1/21104091117.pdf https://ourworldindata.org/safest-sources-of-energy https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(07)61253-7/abstract https://www.nextbigfuture.com/2021/07/2020-fatalities-for-us-roofers-increased-15-as-solar-roof-installations-increase.html -------------------- 69 Conclusion from Studies Remember that in engineering terms, when comparing groups of numbers which contain both both very small numbers and one or more very large numbers, the differences between the small numbers are often not significant. The differences between the small numbers may be the product of our ability to measure these things rather than any real differences. 70 For example, in the article by Ritchie wind power would appear to be twice as dangerous as nuclear. However, the difference between them is 0.02 compared to 24.6 for coal. In other words, the difference between apparently "dangerous" wind and apparently "safe" nuclear is equivalent to 0.08% of the total for coal. It's therefore meaningless and a red herring to even worry about. 71 With the above taken into consideration, generally the different sources of energy fall into two broad categories in terms of number of deaths, injuries, and illnesses. The fossil fuels and biomass fall into one group and wind, solar, and nuclear into another group. 72 Hydro power would seem to fall into the higher risk category or at least somewhere between the two, but this I suspect is mainly due to one exceptionally large dam collapse in China, the Banqian Dam failure in 1975. This is mentioned as being specifically included in the article written by Ritchie. This was a multi-purpose dam, and information on this dam is difficult to find. It is not clear to me whether it had a hydro electric generator associated with either it or another dam that was part of the same system. 73 Some people therefor may argue for its exclusion from the numbers. Of course some people may argue for its inclusion anyway, as it was a dam regardless of whether it actually had an electric generator attached. If we exclude it, then I think the numbers for hydro power would fall into the same range as for nuclear, wind, and solar. 74 Most people would consider hydro power to be safe and clean enough regardless of this and I will rank it as such in any conclusions that I come to. As you can see, even if we have numbers, it can be a matter of opinion as to how to interpret them. -------------------- -------------------- 75 Taking a Systems Approach Now let's take a look at the broader energy picture today and into the future. Many countries in many parts of the world have committed to the concept of "Net Zero", which means eliminating carbon emissions on a net basis. Net zero essentially means the complete electrification of society. We must therefore have electrical energy on demand and at low cost. We must as a result of this look at complete electrical systems rather than individual sources in isolation. 76 At one time many electrical systems were entirely coal or entirely hydroelectric. This is no longer the case. There are now major amounts of wind and solar involved in many countries. However these are inherently intermittent. This means that other sources of energy are inherently also required to have a functional system. 77 If any particular solution inherently requires fossil fuels to meet part of the demand, then the safety, pollution, and climate issues relating to those fossil fuels have to be factored in to that complete system when trying to come up with a relative ranking. Talking about Individual sources in isolation are therefore meaningless in these countries. 78 There are battery systems, but these are mainly used to stabilize and regulate the grid plus to a lesser degree to smooth out short term daily peaks in demand. They do not have the ability to store large amounts of electricity on a large scale for an entire grid for days, weeks, and months to make up for intermittency. 79 So a serious attempt to rank sources of energy would need to look at a variety of representative countries and for each one come up with a plan that involves 'x' megawatts from source 'a', 'y' megawatts from source 'b', etc., and total up the values for each. 80 I am not aware of anyone who has studied this larger issue. However, the problem has to be addressed from this perspective in order for any answer to be useful. Not taking this into account is like ordering a diet soft drink to go with with a high calorie meal and assuring yourself that your plans to diet are fine. 81 This is not to imply there is anything inherently wrong with wind or solar. It does mean that if your goal is to achieve both net zero and a clean environment, you have to look at your entire energy system as a complete system rather than focusing on what you feel are the most reassuring parts of it while ignoring the rest. This does however add to the argument that it is in fact inherently very difficult to come up with a system of ranking energy sources for safety. -------------------- 82 Nuclear, Climate, and Clean Air - Contrasting Examples To give a tangible example we will now look at two different places that followed two divergent paths at roughly around the same time frame. These are the province of Ontario in Canada, and Germany. 83 Ontario had a mix of coal, hydro electric, and nuclear generating plants. Germany had a mix of coal, nuclear and natural gas plants. Ontario shut down their coal fired plants and kept their nuclear plants. Germany however shut down their nuclear plants and kept their coal fired plants. 84 The Phase Out of Coal in Ontario In 2003 Ontario decided to close all of its coal fired generating plants, which consisted of 19 units (that is boilers and turbines) totalling 8,800 MW. This phase out was completed by 2014. 85 Here are the figures for amount of power generated by each energy source in 2003 and 2014. Nuclear went from 42% to 60% Hydro went from 23% to 24% Gas went from 11% to 9% Coal went from 25% to 0% Non-hydro renewable went from 0% to 7%. 86 As you can see, the bulk of that replacement came from increased use of nuclear power. Furthermore, this did not result in simply replacing coal with natural gas. While gas is cleaner than coal, it still has emissions and if you recall from the studies that we looked at earlier, had an estimated death rate roughly 2 orders of magnitude greater than nuclear, solar, or wind. 87 To put this in more practical terms, at one time Toronto regularly had clouds of smog obscuring it, to a large extent due to these coal fired power plants With the phase out of coal, smog days went to zero in 2015 compared to 53 a decade earlier. The 2023 figures for Ontario show carbon emissions of 53 grams per kWh of electricity generated. We can use this as a rough benchmark comparison for total emissions. 88 The Phase out of Nuclear in Germany Until March of 2011, Germany generated one quarter of its electrical power from nuclear. Starting in 2011 however, they began shutting down their nuclear power plants. These were then phased out over the next decade. However, the coal plants were to be kept to 2038. In 2026 Germany began talking about increasing use of coal in order to save gas. In the same year the German chancellor Friedrich Merz stated that the phase out of nuclear was a quote “serious strategic mistake”. EU Commission President Ursula von der Leyen said it was "a strategic mistake for Europe to turn its back on a reliable, affordable source of low-emissions power". 89 I won't go into the details of the phase out, but let's look at some emissions numbers for Germany. If we look at the official numbers from the European Environmental Agency for 2024, for Germany their emissions were 298 grams per kWh of electricity generated. Recall that we are using emissions as a very rough guide to amount of air pollution, and that this has a direct effect on the safety of the overall electrical energy system. 90 So, who actually made their people safer, Ontario who phased out their coal plants and kept their nuclear plants, or Germany who phased out their nuclear plants and kept their coal plants? 91 If you want a comparison directly within Europe, then Germany has one of the highest rates of emissions per kWh of electricity generated, whereas France, who use mainly nuclear power, have one of the lowest at 43 grams per kWh of electricity generated. Again, who is making their people safer, Germany or France? 92 I don't want to make it sound like I am picking on Germany. I am also not going to tell them how they ought to run their country. However they provide a good real world example of how we need to look at things in overall context when we are thinking about the choices that we make. https://www.ontario.ca/page/end-coal https://www.cbc.ca/news/canada/windsor/smog-study-shows-significant-decreases-in-pollutants-in-ontario-1.4151183 https://www.eea.europa.eu/en/analysis/indicators/greenhouse-gas-emission-intensity-of-1 https://world-nuclear.org/information-library/country-profiles/countries-g-n/germany https://www.politico.eu/article/friedrich-merz-is-right-to-reject-germanys-nuclear-phase-out-says-iea-chief-fatih-birol/ https://www.politico.eu/article/germany-considers-ramping-up-coal-power-to-avert-energy-crisis/ https://www.iea.org/countries/estonia/electricity https://www.iea.org/countries/malta/electricity -------------------- 93 Conclusions As we can see, there don't appear to be an abundance of peer reviewed scientific studies that we can simply point to in order to answer the question of safety of all possible major different energy sources once and for all. Collecting the data to even attempt to answer the question is inherently very difficult as we cannot readily conduct experiments to answer the question, and sources of data are not collected or consolidated in a manner which can answer this question adequately. 94 The essence of the problem is that most energy industries are not as tightly regulated and monitored to the same degree that say nuclear power or commercial airliners are, so this data is simply not being systematically recorded. However, a number of people have attempted to make estimates. 95 Their conclusions would seem to be that nuclear, wind, and solar are roughly equivalent in terms of safety. All fossil fuels are much less safe than nuclear, wind, and solar, by as much as several orders of magnitude. 96 We can however say with a reasonable degree of certainty that if a country shut down their nuclear power plants and kept their fossil fuel plants, particularly coal, then they probably made their people less safe than if they had done things the other way around. 97 I hope that I have provided some context in which to think about the issue. Thanks again to brian in ohio for providing the question upon which this episode is based. -------------------- Provide feedback on this episode.
-
-16
HPR4667: UNIX Curio #9 - printf
This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. The echo command is very useful—it prints the arguments given to it, followed by a newline character. (The newline is sometimes also called a linefeed character depending on who is writing or speaking, and has the ASCII decimal value 10.) It has many uses, either in a script or interactively on the command line. The echo utility is used to display text, the value of a variable, or the result of a pathname expansion. It can also feed text to another command in a pipeline. As useful as echo is, it should come as no surprise that it first appeared early on in Bell Laboratories' Second Edition UNIX 1 in 1972. This initial version accepted no options 2 —although the manual page doesn't explicitly say output is followed by a newline character, the description of writing "as a line" seems to imply it. In Seventh Edition UNIX, the manual page 3 makes that clear, and also features the addition of the -n option, which causes echo to print the arguments without a trailing newline character. Eighth Edition UNIX's echo 4 gained the -e option, which allows certain escape codes from the C programming language to be used. These variations caused differences in behavior between different versions of echo . Will running echo -n something on your system output the text "something" without a newline, or "-n something" followed by a newline? Things get even trickier when the command arguments include parameter or pathname expansions. If there are files named "-n" and "something" in the current directory, what does echo * output? Like the previous question, that depends on whether or not your version of echo treats -n as an option. You can't get around this ambiguity by quoting or escaping the "*", because that just causes echo to print a literal asterisk. Example using GNU utilities on Debian 12; both the "echo" utility and the "echo" builtin of bash recognize "-n" as an option. $ ls -1 -n something $ echo * something$ #Shell prompt is on the same line because "-n" was treated as an option to echo $ echo "*" * The solution was to create a new utility, which is the first UNIX Curio for today: printf . This command allows a user to print text similar to the way the identically-named function works in the C programming language. You run printf 5 followed by a format string, followed by zero or more arguments. No newline characters are printed unless specifically indicated by the format string or the arguments. To use printf to print "something" without a newline, that would just be printf something . This demonstrates that you don't need any arguments—in this example, the format string is just a set of regular characters to be displayed. If you wanted a newline character at the end, printf "something\n" would give you that. (In this case, the format string needs to be quoted so the "\n" isn't interpreted by the shell.) In addition to "\n" for a newline, you can also use "\a" for an alert (rings the terminal bell), "\b" for a backspace, "\f" for a formfeed, "\r" for a carriage return, "\t" for a horizontal tab, "\v" for a vertical tab, and "\\" to get a literal backslash. In addition to these special characters, any arbitrary byte can be included using a backslash followed by one to three octal digits; however, it might be difficult to predict what will be output because it can differ based on the character set the terminal is using. It is safer and more portable to stick to the pre-defined characters if possible. The real magic of the printf utility comes from using "conversion specifications" in the format string. Probably the simplest of these to explain is the "%s" conversion specification—it represents a string of any length. The command printf "Hi, %s, how are you?\n" followed by a list of names would print the greeting for each name, putting it in the place occupied by the "%s". $ printf "Hi, %s, how are you?\n" Alice Bob Carol Hi, Alice, how are you? Hi, Bob, how are you? Hi, Carol, how are you? The format string is reused as many times as needed to consume all of the arguments. Take, for example, the command printf "Hi, %s, have you met %s?\n" . If this is run with two name arguments, it would print the sentence on one line, using both names. If run with four name arguments, it would print the sentence twice, once with the first two names and again with the second two names. If you only gave it three names, the last "%s" conversion specification would be replaced with a null string. $ printf "Hi, %s, have you met %s?\n" Alice Bob Hi, Alice, have you met Bob? $ printf "Hi, %s, have you met %s?\n" Alice Bob Carol David Hi, Alice, have you met Bob? Hi, Carol, have you met David? $ printf "Hi, %s, have you met %s?\n" Alice Bob Carol Hi, Alice, have you met Bob? Hi, Carol, have you met ? Three other items can also be given in each conversion specification: flags, the field width, and the precision. The exact meanings of these depend on which type of conversion specifier character you are using. For "%s", using a "-" as the flag causes the text to be left-justified instead of the default right-justified, a field width causes the printed field to be at least as long as the number given, and a precision limits the number of bytes written from the string to the number given. $ #Example of %s with a precision value $ printf "Hi, %.3s, how are you?\n" Alice Bob Carol Hi, Ali, how are you? Hi, Bob, how are you? Hi, Car, how are you? $ #Example of %s with a field width $ printf "Hi, %8s, how are you?\n" Alice Bob Carol Hi, Alice, how are you? Hi, Bob, how are you? Hi, Carol, how are you? $ #Example of %s with a left-justify flag and a field width $ printf "Hi, %-8s, how are you?\n" Alice Bob Carol Hi, Alice , how are you? Hi, Bob , how are you? Hi, Carol , how are you? $ #Example of %s with a left-justify flag, a field width, and a precision $ printf "Hi, %-8.3s, how are you?\n" Alice Bob Carol Hi, Ali , how are you? Hi, Bob , how are you? Hi, Car , how are you? While "%s" is probably the most commonly-used conversion specification, others are available. A whole set of them are dedicated to printing integer values as a signed decimal, an unsigned decimal, an unsigned octal, or an unsigned hexadecimal number. These also can take flags, a field width, and a precision. I think the details and nuances of all this are too complex to clearly explain here, so I will just refer you to the POSIX "file format notation" specification 6 . Be aware that unlike the printf function in the C programming language, the printf utility is not obligated to accept conversion specifications for floating-point numbers. While some implementations might support this, scripts intended to be portable should limit themselves to the restricted set required by the POSIX standard (%d, %i, %o, %u, %x, %X, %c, and %s, plus %b and %% described below). Two more conversion specifications are worth mentioning. The first is only required by the standard for the printf utility, not the C function, and is "%b". This is the same as "%s", except that certain backslash escape sequences in the argument will be treated specially. This includes all the ones described above except for the one using octal digits to represent a byte. In an argument, this is instead represented by "\0" followed by one to three octal digits. An additional backslash escape sequence accepted is "\c"—this does not print anything itself, but causes printf to immediately halt output. The final conversion specification is "%%", which just outputs a literal "%". You can't use a bare "%" in the format string, because printf expects that to introduce a conversion specification. Be careful not to be tripped up by this when trying to print some value as a percentage. Example assuming that the hypothetical "/dev/batterycharge" file on your laptop outputs the battery charge level (42% in this case). As you can see, in some cases an error message might be displayed, but in others it might just behave in a way you didn't intend without complaining. GNU's "printf" utility and the "printf" builtin of bash both support "%e" as a conversion specification as an extension to POSIX. $ cat /dev/batterycharge 42 $ #Wrong $ printf "Your laptop's charge level is $(cat /dev/batterycharge)%.\n" bash: printf: `\': invalid format character Your laptop's charge level is 42$ #Shell prompt appears here from the error $ #Right $ printf "Your laptop's charge level is $(cat /dev/batterycharge)%%.\n" Your laptop's charge level is 42%. $ #Next one treats %e as the specifier, with the space and "l" as flags $ printf "Your laptop has $(cat /dev/batterycharge)% level of charge.\n" Your laptop has 42 0.000000e+00vel of charge. $ #Because no arguments were given, "0" was used for the value to convert Let's go back to the situation I was describing with echo —we have files named "-n" and "something" in the current directory and want to print all their names, separated by spaces. We could do that with printf "%s " * , which would not treat the "-n" as an option. However, the output might look a little weird because there wouldn't be a newline character at the end. We could insert a newline by using "%b" instead of "%s" and following the asterisk with a "\n\c" as the second argument. The "\c" is there to prevent the final space in the format string from being printed after the newline. $ ls -1 -n something $ printf "%s " * -n something $ #No newline was printed here $ printf "%b " * "\n" -n something $ #There's a newline, but also a spurious space before the shell prompt $ printf "%b " * "\n\c" -n something $ #No space before the shell prompt this time Using the "%b" conversion specification can therefore solve one problem, but it also introduces another. Arguments which include a backslash can be interpreted as escape sequences, and many systems are fine with allowing backslashes in filenames. In cases where you're just using the printf utility to display text, it's usually not a big deal if the output looks a little wonky. Where you really need to be careful is when the text is being piped to another program, as control characters and other oddities might cause unexpected results, and can potentially create security problems if processed by a script or utility running as a privileged user. $ #GNU "ls" displays filenames containing a backslash in single quotes $ ls -1 apple banana '\cherry' durian $ printf "%b " * "\n\c" apple banana $ #"\c" in "\cherry" stops output immediately The printf utility looks to have shown up first in 1986's Ninth Edition UNIX 7 , though the earliest manual page I could find 8 is from the Tenth Edition. Its first appearance in BSD seems to be from 1990 in the 4.3 Reno release 9 . Two years later, it was added to Issue 4 of The Open Group's CAE Specification. From what I can tell, it did not seem to be in AT&T's System III—presumably the printf utility did make it into System V at some point but I found it difficult to track this down. While echo is still suitable for use where you know for certain that you want a newline character printed at the end and none of the arguments will start with a hyphen, consider using the printf utility instead for displaying text. It offers more flexibility and features than you are guaranteed to get with echo , although it does require a bit of forethought in constructing a proper format string and arguments. That is not necessarily a bad thing, because a script's author should be thinking about what might happen if it is called with "strange" text or filenames. This episode also provides a good case for being careful when naming files—many filesystems will allow you to use hyphens, control characters, quotation marks, and potentially any character other than a slash or a null byte in a filename. As we've seen, some of these characters can create problems for standard utilities. While it can feel limiting, especially for people not using English, the safest filenames to use on a UNIX-like system consist only of characters in the "portable filename character set" as defined by POSIX 10 and where the first character is not a hyphen. This set includes the lowercase and uppercase letters "a" through "z", the numerals "0" through "9", and the period, underscore, and hyphen. Notably, it does not include the space character. That leads me to another UNIX Curio that I only just now discovered while researching this episode. This is the pathchk utility 11 . It can be run with one or more strings as arguments, checks each one against a set of rules for pathnames, and outputs an error message for each problem found. By default, it checks against the following limits on the system where it's being run: maximum number of bytes in the full path, maximum number of bytes in any component of the path, all byte sequences must be valid in the given directory, and the user running the program must have access to all directories referenced. If run with the -p option, instead of those limits, it checks against POSIX limits: a maximum of 256 bytes in the full path, a maximum of 14 bytes in each component of the path, and each component must only include characters from the portable set. The -P option adds warnings if any component starts with a "-" or if the pathname is completely empty. While the exit status will tell you if the checks succeeded or not, I don't feel like the pathchk utility is well suited to be used in an automated fashion, as the exact wording of its output is not specified and checks cannot be selected individually. However, it can be used interactively to validate pathnames you aren't sure about. See the linked specification for full details. References: A Research UNIX Reader: Combined Tables of Contents https://archive.org/details/a_research_unix_reader/page/n99/mode/1up A Research UNIX Reader: Second Edition UNIX echo manual page (although this page has "v1" typed at the top, the date and the tables of contents indicate it first appeared in v2, a.k.a. Second Edition) https://archive.org/details/a_research_unix_reader/page/n22/mode/1up Seventh Edition UNIX echo manual page https://man.cat-v.org/unix_7th/1/echo Eighth Edition UNIX echo manual page https://man.cat-v.org/unix_8th/1/echo Printf specification https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html File Format Notation specification https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap05.html A Research UNIX Reader: Ninth Edition Table of Contents https://archive.org/details/a_research_unix_reader/page/n95/mode/1up Tenth Edition UNIX echo/printf manual page https://man.cat-v.org/unix_10th/1/echo 4.3BSD Reno printf manual page https://man.freebsd.org/cgi/man.cgi?query=printf&sektion=1&manpath=4.3BSD+Reno Definitions: Portable Filename Character Set https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282 Pathchk specification https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pathchk.html Provide feedback on this episode.
-
-17
HPR4666: How I got into tech
This show has been flagged as Clean by the host. I started out with Basic on the TI-99/4A in 1984. The bare machine could not be programmed by the user in machine code. In 1985 I bought a ZX Spectrum, that gave me total control over the machine. I wrote two FORTH systems on the ZX-Spectrum. In 1988 I got my first 8088 PC, also programming it in FORTH. In 1992 I got an 80386 PC and I ran Linux on it. MCC Interim Release from v. This was the first Linux distro. I have been using Linux ever since. From then on I obtained newer PCs, such as a Pentium in 1995, a Pentium-2 in 1998, a Pentium-4 in 2003 and a Core-2 Duo in 2006. I used several Linux distributions: but I always return to Debian. Links: https://github.com/ForthHub/F83 F83.COM is the ready to run FORTH system. https://github.com/uho/F-PC F-PC - a Forth system optimized for IBM-PC, XT and AT machines running DOS https://www.latte.org/latte.htmlLatte The Language for Transforming Text https://en.wikipedia.org/wiki/Joe's_Own_Editor https://www.freebsd.org/ https://www.debian.org/ https://www.gentoo.org/ https://en.wikipedia.org/wiki/Mac_Mini https://en.wikipedia.org/wiki/PowerPC https://en.wikipedia.org/wiki/Ivy_Bridge_(microarchitecture) https://en.wikipedia.org/wiki/Ubuntu https://en.wikipedia.org/wiki/Firefox https://en.wikipedia.org/wiki/Chromium_(web_browser) https://www.raspberrypi.com/products/raspberry-pi-400/ https://www.intel.com/content/www/us/en/ark/products/series/217838/12th-generation-intel-core-i5-processors.html https://en.wikipedia.org/wiki/Windows_11 Provide feedback on this episode.
-
-18
HPR4665: Pokémon GO
This show has been flagged as Clean by the host. Pokémon Go (stylized as Pokémon GO) is a 2016 augmented reality (AR) mobile game developed and published by Niantic, in partnership with Nintendo and The Pokémon Company, for iOS and Android devices. The game uses GPS to locate, capture, train, and battle Pokémon. It is free-to-play, featuring a freemium model that includes local advertising and offers in-app purchases for additional in-game items. Pokémon Go launched with approximately 150 Pokémon species, with new species regularly introduced. -- Wikipedia.org Provide feedback on this episode.
-
-19
HPR4664: No Input Mixing Tutorial and How to Build a Drone Box
This show has been flagged as Explicit by the host. Some links to other tutorials (in case you need a visual element, these helped me out) Mixing board tutorial https://youtu.be/H-7kQmpjBds Drone Box tutorial https://youtu.be/50C6DBsqy24 Provide feedback on this episode.
-
-20
HPR4663: The hallway track at T-DOSE
This show has been flagged as Clean by the host. T-DOSE TDOSE 2027 Mark you calendars #TDOSE 2027 on 5 and 6 June '27 in the Weeffabriek, Geldrop. T-DOSE Info Booth Hackalot Laptop Revive Free Software Foundation Europe Doeidag and Banray Debian Angry Nerds Podcast Freie Software Freunde - Free Your Model Train Hacker Public Radio: The community Podcast UBports Adfinis Credits The Technical Dutch Open Source Event (T-DOSE) In hpr4641 :: Technical Dutch Open Source Event (T-DOSE) , Ken interviewed Peter van Ginneken about the T-DOSE conference. The Technical Dutch Open Source Event (T-DOSE) is a free conference to promote the use and development of Open Source software. This event has is organised yearly since 2006 in the Brainport region, near Eindhoven, The Netherlands. During this event, Open Source projects, developers and visitors can exchange ideas and knowledge. Peter van Ginneken Opens the Event. We catch up with him at the start of Day 2. Info Booth The backbone of any event is the Info booth and catering. Here we talk to Nick Hibma who when not serving on the Info Booth is treasurer of the T-DOSE organisation. Ready to serve sandwitches, sell T-Shirts, Magic Mugs, and club-mate T-Shirts club-mate Magic Mugs Hackalot Hackalot is the Eindhoven and surrounding area hackerspace. A hackerspace is a place where hackers can work on their own or collaborative projects. You can work and talk together, but you can also do your own thing. Together we can also purchase a lot of cooler tools such as lasercutters and 3d printers. Often there is no suitable place for equipment at home. So if you know someone, you are either an electronics/computer/technical hobby that got out of hand, come on by! Boekenwuurm at the Hackalot stand. The Hackalot stand. [email protected] boekenwuurm.nl Hackalot Laptop Revive Laptop Revive collects discarded laptops, that are still working. We then install Linux Mint to provide a working laptops to students who cannot afford laptops. We are socially involved, sustainable and open. Alex Kok Laptop Revive Laptop Revive Free Software Foundation Europe Free Software Foundation Europe (FSFE) information booth, with information material, stickers and merchandise. Nico was so busy that we were unable to snag an interview this time. However check out our talk with him at the NLUUG Spring Conference 2026 . Free Software Foundation Europe Doeidag and Banray We also interviewed Geert-Jan Meewisse in hpr4639 :: NLUUG Spring Conference 2026 but this time he is here talking about banray.eu In 2025, Meta sold over seven million pairs of camera-equipped glasses that look like regular Ray-Bans. The person wearing them looks like anyone else. But these people are now products, as is everyone they interact with. He then also mentioned the Doeidag project where they encourage people to drop one service at a time on the first Sunday of the month https://doeidag.nl/ https://banray.eu/ Geert-Jan Meewisse Doeidag and Banray Debian The Debian Project is an association of Free Software developers who volunteer their time and effort in order to produce the completely free operating system Debian. Ken Talks to Joost van Baal Llić from the Debian Project Debian Angry Nerds Podcast Angry Nerds, met extra cyber! The Angry Nerds is a Dutch Language podcast about privacy and security It's a live show that is topical and often humorous tech podcast where a group of enthusiastic nerds discusses current technology, IT and cybersecurity topics. The hosts combine technical depth with background conversations, humor and the occasionally a good dose of cynicism. Expect conversations about everything from network infrastructures to software development, from privacy issues to bizarre tech trends. Ken on the Angry Nerds Podcast You can listen to the recording at Angry Nerds op T-DOSE 2026 deel 2 (prikkelarme versie). Angry Nerds Podcast Freie Software Freunde - Free Your Model Train We are a non-profit organization. We are committed to Free Software and Open Standards. Software is not just technology, it's an important part of our daily life. We want to raise awareness of the importance of Free Software and Open Standards. That is why we are concerned with topics outside of technology: politics, education, ethics, psychology, ecology and economics, licenses, ... One of our projects is "Free your model train". Our goal is to raise awareness of the benefits of open standards. Birgit Hücking (@akkolady) standing at the freie-software.org The freie-software.org table with two large train loops, a smaller internal one. Two knitted Tux Mascots. And a lot of information. Close up of the two knitted Tux Mascot. @[email protected] @[email protected] Freie Software Freunde Free Your Model Train https://fymt.de Hacker Public Radio: The community Podcast Hacker Public Radio is a technology focused podcast that releases shows every weekday Monday to Friday. Our shows are created by people like you, and can be on any topic that is of interest to hackers, hobbyists, makers, etc. We are a welcoming community that offers positive feedback and encourages respectful debate. This is our 21st year of operation, and we will release our 5,000th show in August. Everything we do is released under a Free Culture License. We do not vet, edit, moderate or in any way censor any of the audio you submit, we trust you to do that. We will be available to guide you in sharing your knowledge with the community. Having had a stand at FOSDEM (BE), OggCamp(UK), Linux Fest North West(US), Spectrum (FR), we are available to show you how easy podcasting can be. We will be answering your questions, and conducting interviews with anyone with anything interesting to say. The HPR booth. Hacker Public Radio UBports We are developing an open source Linux mobile OS built to be your daily driver... ...and we'd like to welcome you to our community. Next up is a chat with Sander Klootwijk about UBports and Ubuntu Touch. Their website has a list of supported devices . We talk with Sander Klootwijk Proof it's running on actual hardware Yumi The UBports Installer Mascot was not available for comment. Ubuntu Touch on a Fairphone @[email protected] UBports https://devices.ubuntu-touch.io/ Adfinis Accelerate your business with open source-driven automation, security, cloud, and DevSecOps solutions from Adfinis, your end-to-end partner for robust, flexible IT that drives growth and innovation at any scale. Welcome to Our World Full of Open Source At Adfinis, we believe in the transformative power of open source technology to foster innovation, transparency, and collaboration. We are committed to providing solutions free from vendor lock-in, ensuring our clients retain full control and flexibility over their systems. Digital sustainability lies at the heart of our approach, as we strive to create technologies that not only serve the present but also support a long-term, environmentally responsible future. Additionally, we champion digital sovereignty, empowering organizations and communities to own and control their data, infrastructure, and technological destiny. These principles drive us to build a more open, sustainable, and inclusive digital world. Finally we chat to Coen hamers , Robert de Bock , and Annebelle van Waardenburg from Adfinis whose sponsorship made the event possible. https://www.adfinis.com/en/solutions https://www.adfinis.com/en/career Credits Record Needle Rip Free Software Song Provide feedback on this episode.
-
-21
HPR4662: “What Are the Answers I Need, To the Questions I Don't Know Enough to Ask?”
This show has been flagged as Clean by the host. Wandering along the suggestion of topics in Hacker Public Radio, we come to this: how to obtain the answer to something that I don’t even know how to ask? My opinion is: you can’t get an answer without the desire, be it represented by a question or a quest. So, how to pose the correct question? There are a few steps! 1st, interest . Get motivated to know, even if you don’t know exactly what you want specifically. 2nd: be open to deconstruction. To learn is to recognize you didn’t know before (self-pride has to give place for learning). And when confronted with the new, things you thought you knew — even ones you relied on before — may need to be now rejected, if incompatible with the truth you only now receive. 3rd: when the barriers are down, it’s possible to study the theme. Because, at the moment, you don’t know what you don’t know; while one ignores something, he’s only ignorant. One have to listen to others, to practice and /or reflect on the subject until he reaches something specific (and not easily or promptly answerable) worth being the object of his quest or question. 4th: with the question in hand, go for the answer. If it’s valuable for you, more time or less time you are definitely going to get it. If it’s too broad, or you lose interest, or if you need more after getting the initial answer, no problem: you can reformulate and start from the beginning. Great part of the necessary questions only come after the answer already came by a long course of inquiries [questions] (natural consequence of studying with a desire). * * * Credits: 1) Writing is putting thoughts in order and being able to detect what is loosen or unfounded (needing refinement of discard). “The reason so many people have trouble writing is that it’s fundamentally difficult. To write well you have to think clearly, and thinking clearly is hard.” ( https://paulgraham.com/writes.html ) 2) “What Are the Answers I Need, To the Questions I Don't Know Enough to Ask?” ( https://hackerpublicradio.com/about.html#requested_topics ) ( 1 and 2 came at a particular time together and where more deterministic to the line of this episode. ) 3) Teacher Roosevelt, a course administrator (coordinator) of my post-graduation at Escola Paranaense de Direito, that ministered about the end of course paper and how to pose the question you are going to pose in order to write about a topic. 4) Gym exercises (because doing seated leg curls is difficult and gives strange sensations to many men; as thinking , gives a strange feeling one can’t even discern immediately if it’s good, bad, or only odd). Sound effect for credits numbering: “Number pad 1”, by phatkatz4 on Freesound ( https://pixabay.com/sound-effects/musical-number-pad-1-105399/ ) Thank you! Provide feedback on this episode.
-
-22
HPR4661: Laptop Computer Woes, or How I Learned to Love My Tech Hoarding
This show has been flagged as Clean by the host. Blog post: <a href="https://claudiomiranda.wordpress.com/2026/05/19/laptop-computer-woes-or-how-i-learned-to-love-my-tech-hoarding/">https://claudiomiranda.wordpress.com/2026/05/19/laptop-computer-woes-or-how-i-learned-to-love-my-tech-hoarding/ HP ProBook 4540s specifications: <a href="https://support.hp.com/au-en/product/product-specs/hp-probook-4540s-notebook-pc/">https://support.hp.com/au-en/product/product-specs/hp-probook-4540s-notebook-pc/ Tonymacx86 forum post on the 1920x1080 screen upgrade for the ProBook 4540s: <a href="https://www.tonymacx86.com/threads/4540s-lcd-screen-1080p-replacement.88212/">https://www.tonymacx86.com/threads/4540s-lcd-screen-1080p-replacement.88212/ Panelook.com link to the LG156WF1 screen variants: <a href="https://www.panelook.com/modelsearch.php?keyword=LP156WF1&search_id=1839426">https://www.panelook.com/modelsearch.php?keyword=LP156WF1&search_id=1839426 HP ProBook 4540s teardown and screen upgrade video by JSnake Repair: https://youtu.be/3ceZ1m7JFYw?t=1252 Provide feedback on this episode.
-
-23
HPR4660: Robert A. Heinlein: The Future History, Part 1
This show has been flagged as Clean by the host. In his early days as a writer, Heinlein wrote his stories in the context of a shared universe that he called the Future History. These were mostly short stories at first, with hte occasional novella. But they inclode some great stories. The Future History, Part 1 One thing Heinlein became well known for was his Future History. This placed many of his stories in a common framework of a future environment, and allowed events from one story to influence events in other stories. Here is what he had to say about it, in a post I found on the Heinlein Society Facebook site: “I never “created” or “invented” a “Future History.” On April Fool’s Day 1939 I started to write commercially; by the middle of August I had written 8 shorts & a serial. As 5 of these items were more or less to the same fictional background, I found that I was continually having to check back to keep from tripping over my own feet. So I took an old navigation chart, about 3×4 feet, turned it over, made the time scale vertical, then set up 5 columns: stories, characters, technical data, sociological, remarks. Then I checked those first 5 stories, filled data into proper columns at the proper height for the fictional date—and continued to do this with other stories later. The chart was on the righthand wall near my elbow and was unusually messy as I never took the chart down to add to it—just reached over and scrawled on it.” Source: https://www.facebook.com/HeinleinSociety/posts/i-never-created-or-invented-a-future-history-on-april-fools-day-1939-i-started-t/1092968002874634/ One thing that became clear as his Future History developed is that he was not looking at our future exactly. He was very clear in his mind that he was writing fiction, and not issuing prophecies. If you are reading it today, it is best to think of this as a kind of alternate timeline, and this is something that holds true through a lot of his work. Even in his later novels, which were never formally part of his Future History, he would mention events from that past group of works, which may implicitly incorporate them. But this is an area where scholars are in disagreement as to which if the later novels, if any, should be incorporated. And there were unwritten stories that appeared on the chart that would have given further background to the stories that were written. They were stories Heinlein seems to have intended to write at some point, but never got around to writing. You can get more information about this in his book Revolt in 2100. The Future History stories were initially collected primarily in three books: The Man Who Sold the Moon (1950), The Green Hills of Earth (1951), and Revolt in 2100 (1953). Each of them fleshes out this hypothetical world in different ways. The first one, The Man Who Sold The Moon, introduces us to a businessman named D.D. Harriman, who is obsessed with going to the moon. But he thinks it should be done by private enterprise rather than by government. So he concocts a scheme to do this. He promotes a legal theory that the rights to the moon belong to the countries that it directly flies over, sort of like air rights taken to infinity. Then he uses the chaos of competing interests to throw this into the United Nations, and then gets the U.N. to give him the rights. He finally gets to launch a mission to set up a Moon base, but cannot join the expedition because the corporation considers him too valuable to risk. In a sequel story, Requiem, he does get to the moon just in time to die there. Heinlein was never above writing a tear-jerker. Of course, the book has other stories not linked to D.D. Harriman. Heinlein’s fist story, Life-Line, is also collected here. And his second story, Let There Be Light, anticipates the development of solar power panels, but similarly to Life-Line, this earns the enmity of corporate interest, in the form of the Power Syndicate. The Roads Must Roll postulates moving roadways in the future, but the story really is about the sociology of technology in the future. And Blowups Happen, originally from 1940, anticipates nuclear fission as a power source, but it proves to be dangerous. They claim that the craters on the moon were really caused by a series of explosions to reactors that wiped out an earlier civilization. So they move the reactor into space for safety. And this feeds back into The Man Who Sold The Moon when this reactor in space blows up. In these early stories we can already see that Heinlein has a complex view of society. In Life-Line and Let There Be Light corporate power is the villain of the story, and some of this also shows up in Blowups Happen. But in The Man Who Sold The Moon we see that private enterprise is preferred to government action. I think the way this can be reconciled is to see that Heinlein is always concerned with individual personal freedom and opposed to anything that might endanger that, whether from too much government or too powerful corporate interests. The Green Hills of Earth contains the story of the same name, which concerns a former space engineer, Rhysling, now blinded by radiation and unemployable, who is also a poet. And one of his poems has that title. The crew of Apollo 15 named a crater on the moon “Rhysling”, and they planned to read a bit of it at the crater, but those trips could get very busy. Still, as they were getting ready to leave the moon there was this exchange. Note that Allen is the Capcom, and Scott and Jones are the astronauts : “Allen: As the space poet Rhysling (the blind poet in Robert Heinlein’s The Green Hills of Earth) would say, we’re ready for you to “come back again to the homes of men on the cool green hills of Earth.” [Scott – “That’s from the Green Hills of Earth. That’s one we talked about before the flight. Have you read that one?”] [Jones – “Oh, yeah! That was a favorite when I was a kid. Had you read it?”] [Scott – “Sure. (Quoting from memory): We pray for one last landingon the globe that gave us birthTo rest our eyes on fleecy skiesand the cool green hills of Earth.” Although two of the stories in this collection were older, from 1941, most of them are from 1948 and 1949. And there is a reason for that. On December 7, 1941, the United States found itself at war with Imperial Japan, and few days later Nazi Germany. Coming from a family that had fought in every American war you would expect Heinlein to get involved somehow. He could not enlist due to his medical retirement from the Navy, but since he had an engineering background so he became a civilian employee at the Philadelphia Navy Yard, where he was joined by fellow science fiction writers Isaac Asimov and L. Sprague de Camp. A nice retelling of this can be found at Kirkus Reviews, and Asimov also discusses this in his biography. The upshot is that there is a gap of about 5 years when Heinlein did not publish anything. It is also notable that Heinlein by this point had escaped from the pulp science fiction magazines and gotten published in what were called the “slicks’, so-called because the paper they were printed on was slick and higher quality than the pulps. His stories began to be published in places like The Saturday Evening Post, Argosy Magazine, and Town & Country. And these outlets paid higher rates than the pulps, a significant matter for any writer. Heinlein always maintained that the only reason anyone would write was to make money. And the stories were getting to be quite good as well. Delilah and the Space Rigger (1949) tells the story of a woman who joins a construction crew on a space station and faces discrimination, but wins out in the end, which was pretty progressive for the time, but not atypical for Heinlein. Space Jockey is a fairly pedestrian story about a rocket pilot dealing with his every day life. But The Long Watch is an important story to Heinlein’s view of the important things in life. A young officer is assigned to duty on the lunar base, where there are nuclear weapons stored. His superiors want to stage a coup, using those weapons, which can threaten the Earth while being beyond the reach of retaliation. The young officer sacrifices himself to prevent their plot from succeeding, and becomes recognized in a death as a great hero. And this becomes part of the background to a later juvenile novel Space Cadet, as well as being referenced occasionally in other stories, so you can see that he regarded it as an important statement. Gentlemen, Be Seated is a cute little story about a man who saves people when a leak happens in a tunnel on the Moon by plugging the leak with his rear end. The Black Pits of Luna is little thing about a boy scout who is able to rescue his little brother, but it foreshadows the Juvenile novels he later wrote. It’s Great To Be Back! is about a couple who have moved to the Moon, but continually find fault with the living arrangements. They finally decide to go back to Earth, but discover that it was not really the place they had remembered, and they then return to the Moon, which they now realize is home. -We Also Walk Dogs is a gem of a story concerning a company called General Services that basically does things for their clients. Their advertising slogan is “Want somebody murdered? Then DON’T call General Services. But for anything else, call…. It Pays!” They deal a few different problems in this story, but the main one is the development of anti-gravity, and it features a Chinese porcelain bowl. Ordeal in Space is about a spaceman who has an accident that gives him a fear of heights and washed him out of space. But he has to face his fear when he needs to rescue a kitten from the 35th floor. One thing about Heinlein is that he was a firm and devoted cat fancier, so it no accident that a kitten is the one that has to be rescued. And the final story, Logic of Empire, he discusses the development of slavery in the Venus colony as a natural consequence of machinery being expensive and humans being cheap. And in this story there is a background reference to Nehemiah Scudder, who will soon be important in the Future History. One of the things that is worthy of a brief discussion at this point is exemplified by the story Logic of Empire, and that is the reference to the Venus colony. We now know that Venus can best be described as hellish, with crushing air pressure and temperatures high enough to melt metals. The best designed landers can last no more than minutes before being destroyed. But this was not known when Heinlein was writing these early stories. The prevailing view at that time was that Venus was shrouded in clouds because it was very wet and swampy, so that is what Heinlein went with. Similarly his Mars had canals and was inhabited. You just have to go with it in these stories, as you have to do with so much of Golden Age science Fiction, let alone pre-Golden Age. Links: https://www.facebook.com/HeinleinSociety/posts/i-never-created-or-invented-a-future-history-on-april-fools-day-1939-i-started-t/1092968002874634/ https://www.amazon.com/Man-Who-Sold-Moon/dp/0671578634 https://www.amazon.com/Green-Hills-Earth-Robert-Heinlein/dp/0671578537 https://www.amazon.com/dp/B0011GBTKM/ https://www.kirkusreviews.com/news-and-features/articles/asimov-de-camp-and-heinlein-naval-aviation-experim/ https://www.palain.com/science-fiction/the-golden-age/robert-a-heinlein/the-future-history-part-1/ Provide feedback on this episode.
-
-24
HPR4659: Command Line Fun - Recording a show
This show has been flagged as Clean by the host. In this episode Kevie does a step-by-step approach to record an episode of HPR using the FFMPEG tool on the Linux command line. Before beginning please ensure that FFMPEG is installed, it is available in the vast majority of Linux repositories. Start by making a new folder to keep all your files in and move into the new folder (these will be numerous by the end of your recording): mkdir Podcast cd Podcast To start recording audio use the command: ffmpeg -f pulse -i default file01.flac and finish the recording by pressing ctrl+c. I would recommend recording a test piece of audio to ensure that you are recording from your desired microphone and that the levels are to your liking. To listen to the audio file we use ffplay: ffplay file01.flac Once all of the files have been recorded, to reduce the need for editing I would recommend recording several short segments, we need to put these together using: ffmpeg -i file01.flac -i file02.flac -i file03.flac -i file04.flac -i file05.flac -filter_complex "[0:a][1:a][2:a][3:a][4:a]concat=n=5:v=0:a=1" filedone.flac Not that the number of sets squared brackets [] should be the same as the number of files (these start at zero) and the number after n= should be the actual number of files you wish to combine. To remove any extended periods of silence then we can use: ffmpeg -i filedone.flac -af silenceremove=stop_periods=-1:stop_duration=1:stop_threshold=-45dB filefinished.flac Please note that this is a bit flaky at the time of recording (my results have been mixed) and it will re-encode the audio file so never do this with a lossy file such as ogg or mp3 as this will reduce the quality, keep this for lossless versions such as flac or wav files. If you want to spend a bit more time editing the files and getting a better final audio file then the most effective way (but not quick) is to trim the audio from the end and beginning. Listen to the audio files and note the times of any periods of silence. As these normally are at the beginning and end, especially when we are recording in short segments then I will limit the instructions to avoid these becoming silly in length. Firstly clip off the end silences first, if you start with the beginning then it will change the starting position of the end silences. To remove audio from the end of a track use: ffmpeg -i file01.flac -vn -acodec copy -to 00:01:30 file01cut.flac In this example anything after 1 minute and 30 seconds will be removed. The edited audio file will then be saved as file01cut.flac. This method does not re-encode the audio so there is no loss of quality. To remove audio from the start of a track use: ffmpeg -ss 30 -i file01.flac -c copy file01cut.flac In the above example the first 30 seconds of the file will be removed and saved as file01cut.flac Once you have edited each audio file then they will need to be merged together again to make a complete show. Provide feedback on this episode.
-
-25
HPR4658: Audio Revisited
This show has been flagged as Clean by the host. 01 Introduction This is a follow up to my 4 part series on simple podcasting. In this episode I will discuss a number of experiments with audio filtering. These experiments were inspired by comments by listeners and by other discussions about audio on HPR. I am not an audio expert, so I am doing this partly in order to learn something, but mainly in order to have a bit of fun. I hope that you find this entertaining as well. In a comment on the first episode a listener mentioned something called Solocast and said that the method bore a resemblance to the method that I was using. Here is his comment -------------------- 02 Comment #3 posted on 2026-04-03 07:49:58 by Reto It reminds me about Solocast Hi Whiskeyjack, I really liked your podcast and the topic. I cannot remember about your last, but the sound quality of this one was good on my mobile speakers :) The concept reminded me about the program from Norrist (another host on HPR), while similar does it have some differences HPR 3496 https://hackerpublicradio.org/eps.php?id=3496 As I am not on the future feed, I look forward to your next episode. Cheers, Reto -------------------- 03 End of comment. I did not recall having heard the episode on Solocast, but this sounded very interesting. Solocast was in HPR episode 3496 and was released by norrist on the 27th of December 2021. I listened to that episode and does indeed use use the same basic concept of recording short segments of audio and combining them later instead of creating one big recording and editing it with an audio editor. 04 The main difference is that the work flow that I described involves a lot of manual steps, while Solocast is a short Python program that automates the entire process of presenting your script, recording the segments, combining the segments, and filtering and normalizing the result. I won't try to describe Solocast in detail, instead I would recommend just listening to HPR episode 3496 to get norrist's explanation directly. -------------------- 05 While I wanted to make sure that I credited norrist with having come up with this concept four years before I did, this won't be the focus of this episode. Instead I will talk about audio filtering and various experiments that I ran on several different methods. 06 While looking at the source code for Solocast I noticed that it used a filtering method that resembled one used by Jivetalk, a podcast production program that caught the attention of one of the HPR community news presenters. This method involves taking a sample of quiet audio where there is no speaking taking place, and then using this as input to a noise reduction filter which is applied to the voice recording. The filter subtracts the quiet sample from the voice audio, which should theoretically remove the ambient noise. 07 I decided to apply this method to a number of different audio test recordings which were recorded under different circumstances using different hardware. In this way I could see if the method worked equally well under all circumstances or if there were some sorts of noise which it was suited to and some sorts that were not. 08 While I was at it, I also picked several other filter methods to see how they worked as well. Potentially, some methods may be better under some conditions while other methods were better suited to others. -------------------- 09 I won't present all of my experiments, as that would be a bit dull to listen to. Instead I will describe each method and then present audio samples which illustrate my conclusions. There are two pieces of audio software involved, both of which were also used in my series on simple podcasting. 10 The first is Sox, spelled s o x , and which is short for Sound Exchange. Sox is a command line program for audio manipulation. Sox is Free Software, released under the GPLv2 or later. The other is FFMPEG, which is also a command line program. FFMPEG is also Free Software, released under the LGPL V 2.1 or later, and GPL v 2 or later. Sox actually uses FFMPEG for certain operations. -------------------- 11 Audio Hardware For recording hardware I used the following. 12 Maxwell Headset The first is a cheap Maxwell headset that has an electrical noise problem. Unfortunately I don't have a model number for this headset. I described this hardware, the noise problems that I had with it, and how I created filters to deal with the noise in my series on simple podcasting. Briefly though, this is a headset that has a build in microphone on a boom which allows the microphone to be positioned close to the mouth. It connects with a USB cable. 13 Borne Earpiece and In-line Microphone This is a set of earplugs that go in your ears and connected by wires and a very small microphone built into a small bulge in the cable. It connects using a 3.5mm jack. The model number seems to be BUD250-BL. 14 XTrike Headset This is a gaming headset similar to the Maxwell headset described above. The model number is GH-510 It uses a USB connection. 15 Yanmai Condenser Microphone This is a microphone that comes with a small tripod stand. The model number is SF-910 It uses a 3.5mm audio jack. -------------------- 16 This is not a review of the hardware. Rather, I was trying to create audio problems so that I could test ways to fix them. Therefore, do not take the above list as a recommendation of what to buy. However, you can see that I am not using any expensive audio hardware. If you want to make an HPR podcast, you do not need professional level hardware. -------------------- 17 Audio Samples The audio samples are as follows 18 Quiet This was recorded in a quiet environment at my desk. This is my normal podcasting environment and represents optimal conditions. The main reason for this method is to see how the various filter methods perform when dealing with the electrical noise from the Maxwell headset. 19 Small fan This is a small USB powered table fan approximately 10 cm in diameter. It was located roughly 40 cm or less to the left of the microphone, although this varies depending on the microphone. 20 Traffic This was along a busy street with traffic noise in the background. -------------------- 21 Filter Methods Sox noisered Filter with Audio Profile This method uses the Sox noisered filter. Here is a brief quote from the Sox documentation on this filter. Quote Reduce noise in the audio signal by profiling and filtering. This effect is moderately effective at removing consistent background noise such as hiss or hum. To use it, first run SoX with the noiseprof effect on a section of audio that ideally would contain silence but in fact contains noise - such sections are typically found at the beginning or the end of a recording. End of quote For these tests I recorded a separate noise profile to go with each test. -------------------- 22 Basic Manual Filter This is a basic high and low pass filter pair based on the work I had done in my previous series on simple podcasting. However, based on the tests that I have done for this episode, I decided to get a bit more aggressive in terms of filtering. I use a high pass filter of 120 Hz, and low pass filter of 8 kHz. The each filter is then applied twice to increase its effect. I also added band reject filters to deal specifically with 50 and 60 Hz line noise. -------------------- 23 Complex Manual Filter This uses the manually constructed filter described in my series on simple podcasting. This uses the basic manual filter plus a series of custom bandreject filters to fix specific noise problems with the Maxwell headset. -------------------- 24 FFMPEG afftdn Filter The documentation describes this as "Denoise audio samples with FFT." -------------------- 25 FFMPEG arnndn Filter The documentation describes this as "Reduce noise from speech using Recurrent Neural Networks." -------------------- 26 FFMPEG agate Filter I will pronounce this as "agate" for convenience. The documentation describes this as "A gate is mainly used to reduce lower parts of a signal. This kind of signal processing reduces disturbing noise between useful signals." -------------------- 27 Method The experimental method used was to take each noise sample and apply the different filter methods to it. Where there are parameters which can be adjusted, a script was used to generate a series of different sample files with different parameter values. Not all possible parameters were experimented with, as the goal is to see which method produces what sorts of results under different circumstances, not to get the best possible result for the samples that I happen to have. The method in each case was as follows 28 Step 1 Convert the audio file to FLAC if it is not already in that format. 29 Step 2 Apply a basic high and low pass filter described previously to each sample. The reason for this basic filtering is that it eliminates at least some undesired noise in a fairly fool proof manner, leaving less for the more advanced filter to deal with. This should allow for a better test of the filter under realistic conditions. 30 Step 3 Apply the noise reduction filter being tested. 31 Step 4 Normalize the filtered sample to 17 LUFS according to the EBU R128 standard. The EBU standard is described in my series on simple podcasting. Normalizing adjusts the audio signal to a desired loudness level. This allows for more more consistent sound levels and allows us to hear the results under realistic conditions. I normalize the audio individually for each sample as different recording hardware requires different amounts of loudness adjustment. This is different from the typical podcast process where normalizing takes place as the very last step in the process, but it was necessary in this case. 32 Step 5 Concatenate selected sample audio files to one another to allow for better review and comparing. -------------------- 33 Results The results are grouped according to the type of noise which is being mitigated. This allows for easier comparison of the effectiveness of each technique under different circumstances. I have only picked a few examples of interest out of the numerous experiments that I conducted. -------------------- 34 Quiet Recording Environment with Maxwell Headset This compares how well the various filtering methods work on the noise induced by the electronics in the Maxwell headset. This electronic noise consisted of a noise spike every 1 kHz. This should be representative of electronic noise caused by problems in recording hardware. 35 Manual Filter The manual filter applied a narrow band reject filter every 1 kHz from 1 kHz to 12 kHz. This completely removed the otherwise audible whine caused by the noise. 36 FFMPEG afftdn This method allows for setting a noise floor and then specifying how much the noise floor should be reduced by. The method is very sensitive to getting the noise floor correct for that recording. Set the floor too low and nothing happens. Set it too high, and some distortion results. However it seemed to be moderately effective, but it would seem to require checking it and possibly adjusting it each time it is used. 37 FFMPEG agate This method allows setting a noise floor and then suppressing all sound which falls below that level. This method is very sensitive to getting the noise floor correct for that recording. If set too low (or quiet), it is ineffective. If set too high (or loud), it distorts words which come after a pause, which would typically be between sentences. 38 When set correctly, it completely removes noise in the silences between sentences. However, the noise is still audible during speech. This is because the noise in this case is a higher frequency than normal speech, and so stands out more. It may not be a significant problem for noise which is closer to the main vocal frequency band. Overall, this method is not suitable for this particular problem. 39 FFMPEG arnndn This method used the standard model. A variety of different noise reduction models are available. I only tested it with one, std.rnnn It does not seem to introduce much distortion in the voice signal even with a high amount of mix parameter. 40 However, it is only slightly effective at removing the whine from the signal, even with a high amount of mix parameter. Overall, this method does not appear to be useful for this sort of noise problem. 41 Sox noisered Filter This was effective in removing noise between words, but noise can be heard while words are being spoken. It was better than agate however. 42 Overall Conclusion for the Maxwell Headset Noise When dealing with narrow noise bands that occur at known frequencies, the manual filter is leagues ahead of any of the other tested alternatives. 43 Sample Audio Here is a sample audio recording showing the best overall results The sample is repeated, first with only basic low and high pass filtering, and then with the manually constructed filtering. In the first sample you should hear a high pitched background whine. In the second sample, the high pitched whine is completely removed. 44 (Audio sample inserted here.) -------------------- 45 Traffic Noise This was recorded using the Borne in-line microphone connected to a mobile phone while walking along beside a busy street. This was in dry cool spring weather, and the road was paved with asphalt. This should be reasonably representative of podcasting while walking outdoors in a noisy environment. 46 Basic Manual Filter This used the basic manual filter with high and low pass filters. This did nothing very useful in this case as the signal was already filtered within those limits by the recording hardware anyway. The low sample rate of 8 kHz in the phone limited the upper frequency to 4 kHz. Recall that the sample rate has to be twice the highest frequency that you want to detect. Overall, this is not suitable for this sort of problem. 47 FFMPEG afftdn With a high noise floor, background noise is reduced, but not eliminated. There was not much distortion in the voice. This is only slightly useful for this sort of problem. 48 FFMPEG agate With a high threshhold, background noise is reduced, but not eliminated. There was some distortion in the voice. The background noise could also be heard when speaking, but because the frequency of the background signal was similar to the louder voice signal, it was not as noticeable as it would have been if the two were very different. This is moderately useful for this sort of problem. It may be more useful in situations where the background noise was not quite as loud. 49 FFMPEG arnndn With high amounts of noise reduction, much of the background noise is suppressed, but there is not a lot of distortion in the voice. The background traffic noise is still present, but is significantly less. This offers only a moderate improvement. 50 Sox noisered Filter With small amounts of noise reduction voice is clear but traffic noise is present as a very significant continuous warbling sound in the background. This is no improvement on the original and in fact could be seen as making it worse. With moderate amounts of noise reduction, traffic noise is mostly gone, but there are still various squeaks present. Voice is noticeably distorted. With large amounts of noise reduction, traffic noise is gone but voice is highly distorted. This is moderately useful for this sort of problem, but requires careful adjustment. 51 FFMPEG arnndn Followed by FFMPEG agate This combined two different filters. First, it used arnndn to suppress the background noise to a lower level without much voice distortion. Then it applied the agate filter to suppress the noise levels between words still further. This used the same amount of mix and threshold as was found to be most effective when each of these filters was used on its own. The background noise is almost completely gone while distortion of the voice signal is low. 52 Overall Conclusion for Traffic Noise The arnndn combined with agate filters was the most successful at suppressing background noise while limiting the amount of voice signal distortion. 53 Sample Audio Here is an audio sample for what I felt to be the best overall results, the arnndn filter combined with the agate filter. First is the original audio with basic filtering. This is followed with the same audio after being passed through the arnndn and agate filters. 54 (Insert arnndn plus agate audio sample here) 55 Another Sample Here is a second audio sample showing the Sox noisered profile based filter. I have included this to show how a profile based filter can make things worse if you are not careful how you use it. This repeats the test audio 4 times. The first is with basic filtering only. The second uses low amounts of noise reduction. The third uses moderate amounts of noise reduction. The fourth uses high amounts of noise reduction. 56 (Insert noisered audio sample here) -------------------- 57 Small Fan Noise with Yanmai Microphone This was recorded using the Yanmai condenser microphone. A small fan was set up behind and to the left of the microphone. This is intended to represent situations where someone may have a fan or air conditioner running in the background due to hot weather, or has a loud computer fan. 58 A condenser microphone was used for this test as they are more prone to picking up unwanted noise. However, for practical recording purposes, this sort of microphone is unsuitable for this type of environment. 59 Basic Manual Filter This used the basic manual filter with high and low pass filters. This did nothing useful as the fan noise was in the same frequency range as the voice signal. This may be of more help in cases where the noise is below the 120 Hz cut off used in the low pass filter. 60 FFMPEG afftdn With high amounts of noise reduction, much of the background noise is suppressed, but there is some distortion in the voice. The background fan noise is still present, but is significantly less. Overall this is moderately effective. 61 FFMPEG agate This was effective in removing noise between words, but noise can be heard while words are being spoken. However, this was a small voice sample and it is possible that more problems could occur. With less fan noise than was in this sample this technique may work much better. 62 FFMPEG arnndn With high amounts of noise reduction, much of the background noise is suppressed, but there is not a lot of distortion in the voice. The background fan noise is still present, but is significantly less. Overall this was fairly effective. 63 Sox noisered Filter With small amounts of noise reduction voice is clear but fan noise is present as a slight warbling sound in the background. With moderate amounts of noise reduction, fan noise is gone, but voice is somewhat distorted. With large amounts of noise reduction, fan noise is gone but voice is very distorted. 64 In general this method is fairly successful at dealing with this sort of problem. However, there is a trade off between background noise and voice quality. Getting that trade off correct takes experiment and judgment for each specific situation. 65 FFMPEG arnndn Followed by FFMPEG agate This combined two different filters. First, it used arnndn to suppress the background noise to a lower level without much voice distortion. Then it applied the agate filter to suppress the noise levels between words still further. This got rid of virtually all of the background noise between words. If you listen carefully however, there is a slight buzzing sound in the voice signal. 66 Overall Conclusion for Fan Noise with Yanmai Microphone. Of the methods tested, the arnndn followed by agate filter seemed to offer the most improvement for the least effort and least voice distortion. The arnndn filter on its own seemed the next most preferable to me despite leaving some fan noise in the background. 67 Audio Sample Here is an audio sample for what I felt to be the best overall results, the arnndn filter combined with the agate filter. First is the original audio with basic filtering. This is followed with the same audio after being passed through the arnndn and agate filters. 68 (Insert audio sample here) -------------------- 69 Small Fan Noise Recorded with Headset The following is an observation rather than a filtering technique. When a recording was made using the Maxwell headset and listened to on the headset later or with speakers, the fan was virtually inaudible. When the same recording was listened to with the XTrike headset, it was barely audible with careful listening and only identifiable as a fan because I knew it was there. 70 In situations where there is ambient noise, the best noise reduction technique is probably to move the microphone as close to your mouth as possible, although not directly in front of it, and reduce the gain if there is a gain adjustment in the microphone. This will work far better than trying to remove the noise later. If you are recording an HPR episode at a desk, then an inexpensive headset with boom mike may do the job just fine with minimal effort and expense. -------------------- 71 Conclusions I have tested three noise scenarios - Electronic noise in the audio hardware at specific frequencies. Recording outdoors with an inline microphone in a noisy traffic environment. A noisy fan creating background noise in an office. My conclusions on these are as follows. 72 Electronic Noise in the Audio Hardware at Specific Frequencies If you can use Audacity or some other means to find the frequencies which are causing the noise, the best solution, assuming you don't just replace the hardware, is to manually construct filters to remove those specific frequencies. This is the safest solution in terms of only doing what you tell it to and not producing unexpected surprises some time down the road when something changed in the environment. 73 If you are looking for a fairly automatic filtering method, the Sox noisered profile based filter seems to work fairly well. There is an equivalent filter in ffmpeg, but I did not include that in my experiments as it is harder to use in a script because it does not use a separate noise profile file. 74 Recording Outdoors with an Inline Microphone in a Noisy Traffic Environment. In this situation, the FFMPEG arnndn combined with agate filters seem to be the most successful. The Sox noisered filter may work, but at the cost of more distortion in the voice than is seen in the other methods. 75 An inherent problem with any profile based noise reduction method is that if the background noise is not constant, which it seldom is in that sort of environment, the profile may not represent the background noise which is present later on in the recording. This risks adding more distortion in the voice as the profile and later environments diverge. 76 However, for this application a different microphone that provided a better recording would appear to be advisable. A solution which brought the microphone much closer to the mouth and so resulted in a better ratio of voice signal compared to background noise would appear to be necessary, after which the question of what sort of noise reduction to use would need to be re-evaluated. 77 A Noisy Fan Creating Background Noise in an Office. The Sox noisered filter and the FFMPEG arnndn, afftdn, and agate methods all work to some degree. However, they all need correct selection of parameters to achieve the proper results. When I compared all four methods side by side, I found the arnndn combined with the agate filter to be preferable in terms of the trade off between background noise reduction and distortion of the voice signal. The arnndn filter on its own seemed the next most preferable to me despite leaving some fan noise in the background. 78 However, that is a subjective judgment of a specific noise sample when recorded using a specific microphone. Keep in mind though that many listeners will not be listening in an idea environment. They may be doing things where background noise is present rather than in a very quiet room and so may find a small amount of background noise in the recording to be less of a problem than distortion in the voice signal which may make some words harder to understand. 79 When I conducted the same experiment recorded with the XTrike headset I found that arnndn seemed to offer no noticeable improvement. This may be because the amount of audible fan noise was far less with the XTrike headset to begin with. In other words, there is no single best solution here, and you may have to be prepared to try different options to see which one works in your situation. The important thing is to avoid making things worse by applying filtering that is not appropriate for that situation. The best method may be to use a recording method that doesn't pick up the fan noise to begin with. This can include just using a gaming headset with boom mic. 80 I have one final observation on this point regarding headsets. The Maxwell headset has a foam cover over the microphone while the XTrike headset does not. There was some slight audible wind buffeting noise picked up by the XTrike headset that was not observed with the Maxwell. This seemed to cause particular problems with the Sox noisered profile based filter, as this noise was irregular and after filtering would show up as a warbling sound. If you use a headset and plan to use it in conjunction with a fan, it may be advisable to apply some sort of wind cover over it. 81 Combining Complex Filters In several cases I found that combining several complex filters offered better results than using any single one on its own. The basic strategy though is to first use a method which is good at reducing undesirable noise without introducing excessive voice distortion. Then apply a different filter which is good at reducing small levels of background noise to an even lower level while affecting the voice signal as little as possible. This uses the relative strengths of different filter types to compensate for the weaknesses of the other. 82 Different combinations of filters were most effective for different types of problems. I did not try all possible combinations however. Perhaps a further exploration of this would be worth doing in a later podcast. -------------------- 83 Case Study - Noise in Another HPR Episode Audio In the comments to my second episode on Simple Podcasting (which is HPR4618) where I discussed basic filtering, a couple of listeners brought up an interesting point. Antoine mentioned "declicking" in a post. -------------------- Vance replied 84 Antoine, thanks for mentioning the click removal capability in Audacity! While I already knew about its noise removal filter, I wasn't aware it also had click removal. It might have helped me for HPR4637, where some sort of electromagnetic signal was picked up by my microphone/recorder, a Zoom H2 (the tapping sound was *not* present in the room where I recorded). While click removal does seem to distort speech when applied to it (though to my ears, it doesn't sound as weird as when noise removal is done with speech), I could have applied the filter only to the pauses, where the "tapping" is most noticeable. I will consider doing this in the event that I'm not able to eliminate the source of interference in the future, which would be the best way to go. -------------------- 85 End of quote. I found this interesting as it sounded like another audio problem that could be experimented with. I found a sample of the episode which had the clicks and cut a copy of that segment out to experiment with. These sounds are a series of clicks, or "ticks" would be another way to describe them, in the quiet part of the audio between sentences or phrases. 86 Next I used Audacity to study the sound spectrum. I found a massive 60 Hz noise spike. However, my speakers won't reproduce sound that low, and filtering this out didn't reduce the clicks. The clicks turned out to be bursts of noise across the 100 to 800 Hz band, which is right where the main vocal band also is. This makes it difficult to filter based on frequency. The most promising approach would seem to be to filter based on sound level. 87 I tried all of the individual audio filter techniques mentioned in the other experiments above. None produced satisfactory results except for agate, which makes quiet audio quieter. This completely suppressed the clicks. However, when applied to the entire episode it also distorted the start of a few sentences which began with single short syllables. 88 The agate filter has a number of parameters which could be adjusted to try to deal with these cases, although I did not spend the time to do so. Another solution to this distortion problem is to simply not apply the filter to those parts of the audio which are affected. If you record the audio as a series of small individual files, it would be easy enough to filter before concatenating the files together while skipping those files which contain audio which is not suited to this method. Here are the results of the experiments. 89 FFMPEG afftdn This reduces the size of of the ticks, but they are still present. However, they may be reduced to a level which is considered acceptable. 90 FFMPEG agate This was very effective in removing ticks with the right parameters. However, it can introduce some voice distortion in the form of cutting out the start of a few sentences which began with single short syllables. This can be corrected with a very short "attack" parameter to turn off the filter when it detects sound above a set threshhold. 91 FFMPEG arnndn This was relatively ineffective. 92 Sox noisered This was effective in removing the sounds between phrases. However, it introduces some distortion in the voice signal. 93 I also tried combining filters. FFMPEG afftdn Followed by agate This combined two different filters. First, it used afftdn to suppress the background noise to a lower level without much voice distortion. Then it applied the agate filter to suppress the noise levels between words still further. This got rid of virtually all of the background noise between words. 94 Here is a short audio sample from HPR4637. First is the unfiltered audio. Second is the filtered audio using the combined afftdn plus agate filters. Since the "clicks" are very quiet, you may not hear them unless you are in quiet environment. Quite a few listeners would probably not be aware of the perceived audio problem in this episode if it had not been discussed here. None the less, it makes for an interesting experiment. Here it is: 95 (Insert sample audio here) 96 Overall Conclusion for Noise "Ticks" The afftdn combined with agate filters seemed to offer the best overall results when used with the right parameters. However, the author, Vance, speaks very clearly and evenly, and so his voice is ideally suited for use with this filter. Another author's voice may not be as suited to this filter. 97 The Sox noisered profile based filter offers various degrees of trade off between suppressing noise and distorting the voice signal. As to whether this is an acceptable trade off depends on the particular voice in question and how easily understood it is under normal circumstances with out additional distortion. The afftdn filter may be a fairly safe filter to use on its own while producing acceptable if not perfect output. -------------------- 98 Overall Conclusions I have presented only a few of the experiments that I conducted. My overall conclusion after all of this is that there is no universal audio filtering method that works best in all circumstances. There are instead a number of tools in the toolbox, and picking the right one for the job takes a bit of trial and error. 99 However, if you have a repeatable recording environment, then once you have decided what tool you need you should create a script for it so you can have a repeatable processing setup. These conclusions apply to voice podcasting. Music has a different set of criteria and techniques that work well with basic voice podcasting may produce poor results when applied to music which has a broader range of frequency and just as importantly, a broad range of loudness. 100 If you are used to using filters and effects in Audacity, many of the settings on those correspond to arguments in the command line version of ffmpeg. It is worth learning how to use ffmpeg directly to automate your recording process. 101 The experiments that I conducted were greatly assisted by writing scripts which created multiple versions of audio files with different settings, thereby allowing me to try many different alternatives relatively easily. It also allowed me to concatenate different audio samples into a single audio file and so listen to different versions in quick succession, making subjective listening judgments more reliable. 102 It is important to keep in mind in all this that I am playing with audio filtering mainly to have fun. It is not necessary to do any of this if you think your podcast episode sounds just fine without it. So, don't let any of what I have talked about in all this discourage you from simply recording a podcast and sending it in as is. I will include copies of the filters I have described here in the show notes. -------------------- 103 Related Matters Hardware Characterization Using Audio Signals I found it useful to characterize the hardware that I had in order to understand its limitations better before starting the experiments. This involved playing a signal out through a set of speakers and then recording it through a microphone. 104 I used two types of signal for this. One is type of signal is known as a "chirp" signal. This is a sine wave that steadily increases in frequency as it sweeps across the audio spectrum. The standard audio range is 20 Hz to 20 kHz, but for my purposes I limited the upper frequency to 15 kHz to save time as anything beyond that is not very useful for voice podcasts. 105 By recording the chirp signal with a microphone and analyzing it with a Fourier transform, I could quickly see what each device was capable of. See my previous series on simple podcasting for an explanation of what a Fourier transform is and what software to use to see the results of it. Here is a chirp signal. 106 (Insert Audio Sample Here) 107 In addition to a chirp signal, I also used a series of simple tones of specific frequencies. By using these tones of known frequency I could gain an understanding of the limitations of my speakers and headphones, and just as importantly, my own ears. By understanding these limitations I was able to narrow the range of frequencies that I need to deal with quite considerably and set the high and low pass filters accordingly. These tones are a series of flac files generated with ffmpeg. 108 Here is a a sample audio tone at a 2 kHz frequency. 109 (Insert Audio Sample Here) 110 Copies of the script to create the chirp signal and the tones are in the show notes. -------------------- 111 A "Not a Review" of some of the Hardware that I Used I said that I would not do a review of the hardware that I used. However, some of it deserves mention for either how good or bad it was. I will record each section using the hardware being described. 112 Maxwell Headset This is my original recording hardware. This is a headset with boom mic and USB connection. There is no model number on it, so I don't know the model. This probably cost somewhere between 10 and 25 dollars. The earpieces sit on the ears and do not fully enclose them. This makes it light weight and comfortable to wear for extended periods of time. It has a problem however with electronic noise consisting of a noise spike every 1 kHz. I was able to fix this with a series of filters using FFMPEG. Fixing this problem is what got me started in understanding audio. I will probably continue to use this headset to make podcasts. 113 XTrike Headset, Model GH-510 This is also a headset with boom mic and USB connection. I purchased this headset for the purposes of experimentation for this podcast episode. It cost $12.88. I found it to be surprisingly good for the price. It has fully enclosed ear pieces however, which may make it uncomfortable to wear in hot weather. I may try doing some of my future podcasting using this headset. 114 Borne Earpiece and In-line Microphone This is a set of earplugs that go in your ears and connected by wires and a very small microphone built into a small bulge in the cable. It connects using a 3.5mm jack. The model number seems to be BUD250-BL. It cost approximately $3.00. I bought several sets of these and use them for listening to podcasts from an MP3 player. The ear pieces are pretty good for listening with. The microphone works reasonably well when used in a quiet location. It is less good when in a noisy environment. It is very important however to secure the microphone to your lapel or other location reasonably near your mouth and to point the microphone (that is the small hole) outwards and not simply let it dangle freely. If you let it just hang, you will get poor quality and inconsistent audio. 115 Yanmai Condenser Microphone, Model SF-910 I purchased this microphone for the purposes of experimentation for this podcast episode. It cost $3.88. As it is a condenser microphone, it is prone to picking up background noise more and as such is probably not a good choice for podcasting by single person sitting at a desk. However, it is none the less a surprisingly good microphone for surprisingly little money. 116 iCan USB Microphone, Model M-306 I purchased this microphone for the purposes of experimentation for this podcast episode. This has a USB connection. This was also relatively inexpensive at $7.99, or roughly twice the price of the Yanmai microphone. Unlike the Yanmai however, it is absolutely wretched. There was such a high degree of distortion when recording through it that I found I could not use it in the fan experiments which I had bought it for. I ended up buying the Yanmai microphone for that instead. -------------------- 117 Easy Effects Software The techniques described so far all involve recording audio files and then processing them later to produce the desired result. This is probably the simplest and most straightforward way of doing things if you are making a typical podcast. However, there may be instances where you want to apply filtering or other effects on the "live" signal immediately and not after the fact. 118 There is audio software which can hook into your computer's audio system and do this with a live signal. For Linux, there is a package called "Easy Effects". This is Free Software and comes under a GPL V3 or later license. I installed it from the Debian repository under Ubuntu 24.04. 119 You can create various filters and even chain them together to combine them. I played with it a bit but do not know enough about it to discuss it seriously at this time. However, I thought it would be worth mentioning for the sake of those who may wish to try it out themselves. -------------------- 120 Episode Conclusion After having had some fun with audio and listening to other HPR members talk about audio, I thought I would have some more fun by playing with noise reduction filters. I have no intention of becoming an audio professional, but by doing some experiments I learned a few things and had some fun doing it. I hope that the rest of you found this interest as well. I will see you all again later in another episode of Hacker Public Radio. -------------------- Scripts Basic Filter This shows basic high and low pass filters ( 120 Hz and 8 kHz respectively) and band reject filters for 50 and 60 Hz. # The high and low pass filters. hlpfil="highpass=f=120, highpass=f=120, lowpass=f=8000, lowpass=f=8000" # Band reject filters filter for 60Hz and another for 50Hz. linefil="bandreject=f=60:width_type=h:w=20, bandreject=f=50:width_type=h:w=20" # Filter using ffmpeg. ffmpeg -i inputfile.flac -af "$hlpfil, $linefil" outputname.flac # ====================================================================== afftdn Filter # noisefloor should be between 20 and 80. noisefloor=$1 # Run the noise reduction. ffmpeg -i testrec-filtered.flac -af "afftdn=nr=10:nf=-""$noisefloor" tmptestrec.flac # ====================================================================== agate Filter # threshold shoud be between 10 and 80. threshold=$1 # Run the noise reduction. ffmpeg -i testrec-filtered.flac -af "agate=threshold=-"$threshold"dB:range=-60dB" tmptestrec.flac # ====================================================================== arnndn Filter # mix should be between 0 and 1. mix=$1 # Run the noise reduction. ffmpeg -i testrec-filtered.flac -af 'arnndn=model=std.rnnn:mix='"$mix" tmptestrec.flac # ====================================================================== sox noisered Filter # Generate the noise profile from a sample of background noise. sox silencefiltered.flac -n noiseprof noise.prof # nramount shoudl be between 0 and 1 sox testrec-filtered.flac noiseout-testrec.flac noisered noise.prof "$nramount" # ====================================================================== Manual Filter for Maxwell Headset Noise # Create a series of band reject filters, from 1 kHz to 11 kHz. ftemplate="bandreject=f=%s000:width_type=h:w=100" kilospikefil=$( seq 1 11 | xargs printf "$ftemplate," ) # Using ffmpeg ffmpeg -i testrec-filtered.flac -af "$kilospikefil" tmptestrec.flac # ====================================================================== Create a "chirp" signal # Start frequency. f0=20 # End frequency. f1=15000 # Duration of signal. duration=10 ffmpeg -f lavfi -i "aevalsrc=sin(2 * PI * (0.5 * ($f1 - $f0)/$duration * t^2 + ($f0 * t))):s=44100:d=$duration" -c:a flac -af "aformat=sample_fmts=s16" chirp.flac # ====================================================================== Generate Audio Tones toneout () { printf -v freqval "%05d" $1 ffmpeg -f lavfi -i "sine=frequency=$freqval:duration=3" tmptone.flac # Normalize ffmpeg -i tmptone.flac -af loudnorm=I=-17:TP=-2.0:LRA=4.0 -ar 44.1k -sample_fmt s16 tone$freqval.flac rm tmptone.flac } # List of frequencies in hertz. freqlist="50 60 100 120 130 140 150 160 170 200 500 1000 2000 3000 4000 5000 6000 7000 8000 9000" for freq in $( echo $freqlist ); do toneout $freq done # ====================================================================== Provide feedback on this episode.
-
-26
HPR4657: UNIX Curio #8 - Comparing Files
This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. Most users of UNIX-like systems are probably familiar with the diff utility. It is widely used with source code to compare two files and see what the differences are between them. Non-programmers, like me, also use it to examine what has changed in different versions of scripts or configuration files. Quite a few pieces of newer software can compare different versions of data and express changes in a format either identical to or similar to diff output. However, there are two other long-standing tools for this purpose that are far less known and deserve in my view to be termed UNIX Curios. The first of these is cmp 1 . While diff is primarily intended to be used on text files and compares them line by line, cmp compares files byte by byte. In my experience, its main use is to see whether two binary files are in fact identical—if they are, cmp outputs nothing and returns an exit status of 0. Back when methods of transferring files were not as reliable as they are today, this was a tool I would reach for sometimes. For example, you could use it to confirm that the data on a CD-ROM you burned was the same as the original. If there is a difference between the files, cmp will return an exit status of 1. By default, it will also print the location (byte and line number) of the first differing byte. When used with the -l option, it will print the location and value of every byte that differs. There is one exception to these: if the files are the same except that one is shorter than the other, it will print a message to that effect. The exit status will still be 1 in that case. Using the -s option with cmp will cause it to be totally silent and output nothing. Only the exit status will indicate whether the files are the same, different, or if the exit status is greater than 1, that an error occurred. This makes it useful for scripting, for example in case you wanted to confirm that a file copied to another location arrived fully intact. It is worth noting that diff is also capable of comparing binary files—however, it is not required by POSIX to report what is actually different or where differences occur. The same exit status as in cmp is returned: 0 if the files are the same, 1 if they are different, or greater than 1 if an error occurred. While many implementations offer an option to suppress the output, this is not in the standard 2 so the most portable method would be to instead redirect output to /dev/null . On my system the diff utility is three times the size of cmp , so if you don't need its extra capabilities, it is a less efficient way of doing the job. The other UNIX Curio for today is comm , and this utility 3 is also intended to compare two files to see what is common between them. Ken Fallon briefly talked about it a few years ago in HPR episode 3889 . Compared to the others, it has a much more specific use case. The two files are expected to be text files that are already sorted. What comm will do is print a tab-separated list of all the lines appearing in either or both files. Lines only in the first file will appear in the first column, lines only in the second file will be in the second column, and lines in both files will be in the third column. Any combination of the options -1 , -2 , and -3 can be used with comm to suppress printing of the first, second, or third column respectively. Using all three options at the same time is supported but it results in no output, so that isn't very useful. Unlike the other utilities, the exit status of comm doesn't tell you anything about the two files. It will be 0 if the program ran successfully, and greater than 0 if it didn't. I'm not sure if I have ever actually used comm for anything practical. I find its default output a bit difficult to meaningfully interpret, plus you need to ensure the two files are already sorted. It seems to be best suited to comparing lists, and one use case that Ken Fallon mentioned would be comparing two lists of files to see if any are missing. The command comm -3 listA listB would print files that only appear in listA in the first column and those only in listB in the second column. This would let you ignore all the filenames that appear in both and focus on those that were absent from one or the other. If on the other hand you only wanted to see the filenames that are on both lists, comm -12 listA listB would give you that. Some more frivolous potential uses also come to mind. If for some reason the cat utility is broken on your system, you could use comm listA /dev/null to print the file listA instead. If you want to insert tab characters before every line of a file but have an aversion to using sed or awk , then comm /dev/null listA would output listA with one tab before each line, and comm listA listA would insert two tabs. A bit silly, but it would work. The GNU implementation of comm even lets you choose something other than a tab to separate the columns 4 , so you could go wild with that. According to the POSIX specifications for cmp and comm , one of the two filenames given as arguments, but not both, can be a " - ", in which case standard input will be used for that "file" in the comparison. Also, the results are undefined if both arguments are the same FIFO special, character special, or block special file. Some implementations might not have these limitations, but you shouldn't rely on that everywhere. All three of these were developed quite early. The cmp utility appeared in 1971's First Edition UNIX 5 , while comm and diff seem to have made their debut in Fourth Edition UNIX 6,7 from 1973. The original versions might not have behaved exactly like their modern counterparts, and newer implementations (especially of the diff utility) have acquired additional options and capabilities, but the basic operation of each has stayed the same. The next time you need to compare files against each other, consider whether cmp or comm might be appropriate before automatically reaching for diff . They all have their uses in different situations. References: Cmp specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/cmp.html Diff specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/diff.html Comm specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/comm.html GNU coreutils manual: comm https://www.gnu.org/software/coreutils/manual/html_node/comm-invocation.html First Edition UNIX cmp manual page http://man.cat-v.org/unix-1st/1/cmp Fourth Edition UNIX comm manual page https://www.tuhs.org/cgi-bin/utree.pl?file=V4/usr/man/man1/comm.1 Fourth Edition UNIX diff source https://www.tuhs.org/cgi-bin/utree.pl?file=V4/usr/source/s1/diff1.c Provide feedback on this episode.
-
-27
HPR4656: My review of musicozy sleep/exercise bluetooth headband
This show has been flagged as Clean by the host. Sound quality, ease of use, comfort, battery life, price https://themusicozy.com/products/headband/index.htmlProvide feedback on this episode.
-
-28
HPR4655: Tips for Glasses
This show has been flagged as Clean by the host. HPR and Glasses HPR community members are nerds We know from movies that nerds wear glasses We can conclude that HPR community members wear glasses Cleaning Background I have very oily skin and am constantly smudging my glasses All lenses wipes are too small. I accidentally touch the lenses with my fingers when I am trying to clean them And smudge them up again I heard an eye doctor say, the best way to clean glasses is with dish soap and dry with a microfiber cloth I tried this method It is a great solution because you wash glasses and fingers tips at the same time Cleaning Method Hand soap is OK to use on glasses. get the soap on you fingertips Then hold your glasses one of the lenses Use your thumb and finger tips to clean the glasses the soap removes the excess oil from your fingertips no smudges when you dry with the microfiber cloth Progressives With regular progressives the top of the lenses is for distance vision and the bottom of the lenses is for up close vision There is a zone in the middle for medium distance, but it is very narrow Using a computer at work I kept having to move my head up and down to find the best position for working. Computer Glasses have more space for intermediate distances From https://iotlenses.com/blog/eight-types-of-progressive-lenses.html Computer progressive lenses, also known as occupational designs or enhanced readers, are optimized for intermediate and near vision, making them ideal for tasks like prolonged computer work or activities at mid-range distance Provide feedback on this episode.
-
-29
HPR4654: What's in my component Box?
This show has been flagged as Explicit by the host. Background This episode describes the contents of an old fishing tackle box I got as a birthday present very many years ago. I used it over the years to gather a pile of mostly useless electronic components and junk. I came across it recently when we were doing a clear out. Mrs X wanted to throw it out. I had a lot of sentimental value due to the long time I’ve had it. I am a bit of a hoarder and it took a bit of persuasion on my part to be able to keep it especially since I could not think of a single use for any of the contents! I’ve not had time to add links against all the items. Hopefully I’ve managed to add links to most of the more unusual items. Apologies if I’ve missed any. Contents of Box Top row left to right, back to front Compartment 1 Stereo Jack plug & socket Double pole double throw (2-way) switch 12V dc relay from RS 3 legged semiconductor (voltage regulator?) Compartment 2 Multiple fuses mostly for cars Compartment 3 Small silica jelly pouch Micro Switch Reed switches (Not reed relays as I call them in my show) Compartment 4 Potentiometers, Rheostat, variable resistors https://en.wikipedia.org/wiki/Potentiometer Control knobs Compartment 5 Large selection of miniature panel mounted toggle switches https://en.wikipedia.org/wiki/Switch#Toggle_switch Compartment 6 Two crocodile clips https://en.wikipedia.org/wiki/Crocodile_clip Half of a Hi Fi speaker plug https://en.wikipedia.org/wiki/File:Speaker_din_male_and_female.jpg Washer Compartment 7 Mostly chocolate block connectors https://en.wikipedia.org/wiki/File:Leuchterklemme_75dpi.jpg Small selection of glass cartridge Fuses Single 13 Amp mains fuse for UK plug Assortment of soldering iron tips Crocodile clip Air adapter for foot pump for blowing up an air bed or similar Spare nozzle for a de-soldering tool. Made from PTFE https://en.wikipedia.org/wiki/Desoldering#/media/File:Vacuum_plunge.jpg https://en.wikipedia.org/wiki/Polytetrafluoroethylene Ferrite ring https://en.wikipedia.org/wiki/Ferrite_bead Compartment 8 Rubber Grommets and a heatsink Middle row left to right, back to front Compartment 1 Large selection of glass cartridge fuses Compartment 2 Another large selection of glass cartridge fuses Small selection of diodes, some quite Large https://en.wikipedia.org/wiki/Diode Compartment 3 Two multi turn precision variable resistors https://en.wikipedia.org/wiki/Potentiometer#/media/File:Beckman_Helipot_potentiometer_SA1400A_2007.075.002.jpg Cheap variable resistor with in built switch Compartment 4 Bag of multi coloured LED’s (Cheap for Pi add on board) 2 inch 8 ohm speaker Compartment 5 Panel mounted heavy duty toggle switch, two pole one way Compartment 6 Light Dependent Resistor (LDR) https://en.wikipedia.org/wiki/Photoresistor Large Tricolour LED Compartment 7 Variable capacitors PCB (Printed Circuit board) mounted Spare nozzle for de-soldering tool. Made from PTFE Light Dependent Resistor (LDR) Panel mounted heavy duty rotary switch, two pole one way Bag of various fuses Halogen 12V car headlight bulb Variable resistor with long plastic spindle, panel mount Heatsink possibly for heavy duty transistor Compartment 8 Variable resistor PCB (Printed circuit board) mounted Bottom row This is just a big jumble of stuff, listed in no particular order DC red and black power lead with inbuilt fuse Small solder-less breadboard for prototyping https://en.wikipedia.org/wiki/Breadboard Bag of various plugs & connectors mostly Jack plugs and sockets Bag of quartz crystals and a components receipt from 1985 I added crystals to an old PMR (Private mobile radio) boot mount radio. Possible a Pye Westminster Radio. I looked but couldn’t find picture of the radio Several bits of Perfboard or strip board https://en.wikipedia.org/wiki/Perfboard 1 Watt audio amplifier kit with instructions not built! D-type serial port mounted to PCB with screw down connections Bag of resistors https://en.wikipedia.org/wiki/Resistor Large bag of LED panel mount holders Electronic Instruction guide XH79L (Maplin) price 25p Chocolate block connecting block https://en.wikipedia.org/wiki/File:Leuchterklemme_75dpi.jpg Telephone pickup for recording phone calls. (Never worked) https://www.amazon.co.uk/Black-Telephone-Pickup-Sensitive-Microphone/dp/B000L1OZG8 Multi coloured ribbon cable Plastic trimming tool screwdriver set. https://www.amazon.co.uk/C-K-T4857-Precision-Plastic-Trimmer/dp/B000WKIYTW Panel mounted BNC socket (Baby Neill Constant) https://en.wikipedia.org/wiki/BNC_connector Provide feedback on this episode.
-
-30
HPR4653: Starting the Habit of Reading
This show has been flagged as Clean by the host. Openreads is a privacy oriented and open source cross-platform app written in Flutter available for Android and iOS for keeping track of your books. There are four lists provided so you won't get confused: books you finished, books you are currently reading, books you want to read later, books you didn't finish. You can use custom tags and filter through them. A book can be added by: looking it up in the Open Library database, scanning its barcode, adding its details manually. You can also view some cool statistics! Links: Battletech: Mercenary's Star by William H. Keith Permaculture for the Rest of Us by Jenni Blackmore Piranesi by Susanna Clarke Battletech: The Price of Glory by William H. Keith The Urban Homestead by Kelly Coyne Battletech: The Sword and the Dagger by Ardath Mayhar Battletch: Warrior En Garde by Michal A. Stackpole Sisterhood of Dune by Brian Herbert and Kevin J. Anderson Fundamentals of the Wudang Sword Method by Scott M. Rodell Battletech: Warrior Riposte by Michal A. Stackpole <a href="https://www.goodreads.com/book/show/26087493-the-cardio-code"> The Cardio Code by Kenneth Jay Mentats of Dune by Brian Herbert and Kevin J. Anderson How to Grow More Vegtables by John Jeavons Provide feedback on this episode.
-
-31
HPR4652: simon says
This show has been flagged as Clean by the host. From https://en.wikipedia.org/wiki/Simon_(game) Simon is an electronic game of short-term memory skill invented by Ralph H. Baer and Howard J. Morrison, working for toy design firm Marvin Glass and Associates, with software programming by Lenny Cope. The device creates a series of tones and lights and requires a user to repeat the sequence. If the user succeeds, the series becomes progressively longer and more complex. Once the user fails or the time limit runs out, the game is over. The original version was manufactured and distributed by Milton Bradley and later by Hasbro after it took over Milton Bradley. Much of the assembly language code was written by Charles Kapps, who taught computer science at Temple University and also wrote one of the first books on the theory of computer programming. Simon was launched in 1978 at Studio 54 in New York City and was an immediate success, becoming a pop culture symbol of the 1970s and 1980s Provide feedback on this episode.
-
-32
HPR4651: HPR Community News for May 2026
This show has been flagged as Explicit by the host. New hosts There were no new hosts this month. Last Month's Shows Id Day Date Title Host 4630 Fri 2026-05-01 Playing Civilization V, Part 11 Ahuka 4631 Mon 2026-05-04 HPR Community News for April 2026 HPR Volunteers 4632 Tue 2026-05-05 Hackerpublic Radio New Years Eve Show 2026 Episode 6 Honkeymagoo 4633 Wed 2026-05-06 Ham Radio Licence Lee 4634 Thu 2026-05-07 Upgrade Failsause operat0r 4635 Fri 2026-05-08 What did I do at work today? Part 3 Section 1 Lee 4636 Mon 2026-05-11 7 seconds memory Antoine 4637 Tue 2026-05-12 UNIX Curio #6 - at and batch Vance 4638 Wed 2026-05-13 Simple Podcasting - Episode 3 - Analyzing and Filtering Whiskeyjack 4639 Thu 2026-05-14 NLUUG Spring Conference 2026 Ken Fallon 4640 Fri 2026-05-15 Robert A. Heinlein Ahuka 4641 Mon 2026-05-18 Technical Dutch Open Source Event (T-DOSE) Ken Fallon 4642 Tue 2026-05-19 Hackerpublic Radio New Years Eve Show 2026 Episode 7 Honkeymagoo 4643 Wed 2026-05-20 HPR Beer Garden 13 - Triple IPA Kevie 4644 Thu 2026-05-21 Response to comments on HPR4424: Newsboat... Archer72 4645 Fri 2026-05-22 ZERO HOUR: FRIDAY AFTERNOON APK HACKING operat0r 4646 Mon 2026-05-25 Mobile Gaming Elsbeth 4647 Tue 2026-05-26 UNIX Curio #7 - Compression Vance 4648 Wed 2026-05-27 Simple Podcasting - Episode 4 - Audio Analysis Fun Whiskeyjack 4649 Thu 2026-05-28 What did I do at work today? Part 3 Section 2 Lee 4650 Fri 2026-05-29 Playing Civilization V, Part 12 Ahuka Comments this month Past shows hpr4424 (2025-07-17) "How I use Newsboat for Podcasts and Reddit" by Archer72. أحمد المحمودي said: "How did I find HPR" (2026-05-12 17:16:07) candycanearter07 said: "Re: How did I find HPR" (2026-05-12 19:36:04) hpr4502 (2025-11-04) "Cheap Yellow Display Project Part 3: Reverse beacon network " by Trey. Archer72 said: "Morse code" (2026-05-19 15:00:39) hpr4567 (2026-02-03) "Movie Recommendations for Hackers" by Deltaray. Antoine said: "Some watched!" (2026-05-27 03:33:12) hpr4587 (2026-03-03) "UNIX Curio #1 - Shell Archives" by Vance. Dave Morriss said: "Great reminder! I had forgotten shar" (2026-05-07 18:18:05) Vance said: "Color printing" (2026-05-08 22:02:23) hpr4607 (2026-03-31) "UNIX Curio #3 - basename and dirname" by Vance. Vance said: "Correction" (2026-05-12 01:03:12) hpr4618 (2026-04-15) "Simple Podcasting - Episode 2 - Basic Filtering" by Whiskeyjack. Henrik Hemrin said: "Generating sine wave" (2026-05-14 19:59:46) Whiskeyjack said: "Reply to Henrik Hemrin on Sine Waves in HPR4618" (2026-05-15 15:22:16) hpr4627 (2026-04-28) "UNIX Curio #5 - Faster, Pussycat! Kill! Kill!" by Vance. Steve Barnes said: "Thanks for the context!" (2026-05-21 05:28:05) hpr4628 (2026-04-29) "Nuclear Power Technology Follow Up" by Whiskeyjack. Antoine said: "o/" (2026-05-17 00:24:23) Whiskeyjack said: "Reply to Antoine on HPR4628" (2026-05-17 18:08:18) This month's shows hpr4631 (2026-05-04) "HPR Community News for April 2026" by HPR Volunteers. Whiskeyjack said: "Response to Ken Fallon in HPR4631 Community News" (2026-05-04 05:13:02) candycanearter07 said: "new episodes" (2026-05-04 18:47:39) Ken Fallon said: "new candycanearter07 episodes" (2026-05-05 09:00:00) أحمد المحمودي said: "Thanks for the encouragement" (2026-05-12 17:18:20) hpr4634 (2026-05-07) "Upgrade Failsause" by operat0r. YourName said: "Why is the audio so bad OMG sorry" (2026-05-06 20:05:43) Ken Fallon said: "Already reported and fixed" (2026-05-06 20:35:49) hpr4637 (2026-05-12) "UNIX Curio #6 - at and batch" by Vance. candycanearter07 said: "still useful!" (2026-05-13 03:43:37) norrist said: "at for scheduled reboots" (2026-05-16 19:41:04) Vance said: "Good points" (2026-05-17 03:03:47) Whiskeyjack said: "At and batch in HPR4637" (2026-05-17 18:35:11) hpr4640 (2026-05-15) "Robert A. Heinlein" by Ahuka. Antoine said: "Nice tips" (2026-05-17 00:22:01) Kevin O'Brien said: "I enjoyed doing it" (2026-05-17 19:14:19) hpr4646 (2026-05-25) "Mobile Gaming" by Elsbeth. candycanearter07 said: "my opinion of mobile gaming" (2026-05-27 00:24:58) hpr4647 (2026-05-26) "UNIX Curio #7 - Compression" by Vance. xmanmonk said: "Great Series" (2026-05-27 00:29:32) candycanearter07 said: "thoughts" (2026-05-27 13:20:13) Mailing List discussions Policy decisions surrounding HPR are taken by the community as a whole. This discussion takes place on the Mailing List which is open to all HPR listeners and contributors. The discussions are open and available on the HPR server under Mailman. The threaded discussions this month can be found here: https://lists.hackerpublicradio.com/pipermail/hpr/2026-May/thread.html Events Calendar With the kind permission of LWN.net we are linking to The LWN.net Community Calendar. Quoting the site: This is the LWN.net community event calendar, where we track events of interest to people using and developing Linux and free software. Clicking on individual events will take you to the appropriate web page. Provide feedback on this episode.
-
-33
HPR4650: Playing Civilization V, Part 12
This show has been flagged as Clean by the host. In our sample game we look at playing as Austria and aiming for a Diplomatic Victory. And our focus is on puppeting Citty-States, but be I misunderstood and instead of making a Diplomatic Victory easier, it makes it harder. I still managed to get my Diplomatic Victory, but a Science or Domination Victory would definitely have been easier in this scenario. Playing Civilization V, Part 12 A Diplomatic Victory Strategy Civilization V introduced a new Victory type and I thought it might be fun to try this strategy for a sample game to see broadly how this would work. I decided I would play as Austria on Prince level, which means that all players are equal and no one is favored. For my map I chose Fractal, Map Size = Standard, Game speed = Standard. My only Advanced Option was Quick Combat, because I didn’t need to see that drawn out, particularly since I plan to avoid combat as much as necessary. My leader is Maria Theresa, and her Austria has a unique ability called Diplomatic Marriage. This allows us to either Annex or Puppet any city-state that has been allied to us for 5 turns with the proper expenditure of Gold. My plan was to use this to Puppet the city-states to control their votes for the Diplomatic Victory, but that was a misunderstanding. I actually implemented one of the hardest ways to win a Diplomatic Victory. But that is what I did. Now to get them to ally with me the most effective way to do this is with cash and lots of it, and of course even more cash to actually effect to Puppeting of them. So my overriding objective in this game is to amass a large Treasury. But of course I cannot ignore my military either, since a weak military invites attacks form greedy neighbors. And I may need to “liberate” the occasional city-state if another player conquers them. Austria also has a Unique Unit, the Hussar, which replaces the Cavalry unit. It can move after attacking, has a flanking bonus, and has one extra movement. And the Unique Building is the Coffee House, which increases the generation of Great People in the city by 25%. So you can expect me to build these in every city as well. With that in mind, I started the game and settled in place, I had Mountains nearby, but also Sheep and Silver within my city, so some useful resources. I immediately started to produce a Scout as my first unit, and sent my Warrior out to explore. In the very early stage I focus on exploring the surrounding area and finding any Goody Huts, i.e. Ruins. My initial city site is not exactly ideal, as it is all hilly with Jungle nearby. After building my two Scouts, my next priority was to build a Worker unit to increase the productivity of my city. And for my first social policy I unlocked Tradition. When I got my second policy I picked Oligarchy, and plan to complete all of the Tradition tree. I cleared out a Barbarian encampment, and then discovered my first City-state, Vilnius. Then I needed to clear out another Barbarian encampment that was blocking me from finishing my exploration. Meanwhile I am focusing on getting techs for sailing the ocean blue, because that is how I plan to get trade routes, discover City-states I can puppet, and so on. Because money is key to my strategy I made a beeline for Currency in the my Science research. And while headed there I completed the Tradition tree. When I can get there I will work on the Commerce tree to maximize my cash, but until then the Patronage tree will let me improve my City-state relations, which is important for improving my relationships with City-states. After all, you need to be allies with them for 5 turns before you can puppet them. I now have three cities, and can probably squeeze out a few more, which should be sufficient to my needs. At Turn 141 I have 6 cities, which is all I will build in this game. I now have the technology to build Workshops, which are the first productivity boosters available, so I set all my cities to building them. For Research my immediate object was to get to Compass so I could build the Galleass, which would let me do more ocean exploration. But to go into deep Ocean I will need to go further to get Astronomy, which will let me build the Caravel which can enter deep ocean. Once I got that I switched to Banking, not just for the money, but as a prerequisite to building the Forbidden Palace, which grants two additional delegates in the World Congress/United Nations. When I get Banking I’ll go back to Astronomy, and then Navigation, to advance my seagoing capabilities. By Turn 216 I had gotten Banking and started on the Forbidden Palace. And by luck, just as I got started I got a Great Engineer. I am holding him in reserve in case I need to hurry production, since Great Engineers are the only way to do that in Civ 5. I also picked up Astronomy, which will let me build Caravels to explore the whole ocean. My next research priority will be Gunpowder since it is time to beef up my defenses. At Turn 240 my Caravels started to come out, and I found several new City-States. And since my Treasury is healthy (I started with 6,000 gold, and I’m bringing in 100 each turn), I began the process of puppeting the City-states. Puppeting City-states as Austria Let’s look at this in detail since it is important. The requirements are 2 things: Be allied with the City-state for 5 consecutive turns Have the cash needed. This amount is not too much early on, but it rises over time. So, how do you become allies with a City-state? There are a number of things you can do to improve your relationship. You can take on a quest that a City-state has published, which can be things like “Find another Natural Wonder” or “Create a Great Admiral”. These quests pop up continuously throughout the game, and you are free to ignore them, but fulfilling one will improve your relationship. Trade will also improve your relationship, so in this game all of my Trade routes were made with City-states. If you are in a position to have a successful war, you can find a former City-state that was conquered by one of the other Empires, liberate it, and then it will be your ally for the rest of the game. Giving them presents is how I usually do it, though. You can give them units or money. If your purpose is to get allies, money works best. But I do gift units in 2 circumstances. First, if I have obsolete units, giving them away might be better then deleting them. Second, if a City-state is under attack by another Empire. Gifting them units might help them hold out and make life difficult for a rival. The place where all of this is done is the City-state screen which opens up when you click on the bar above the City-state, which is where you handle all of your relationships. At the top of this screen you see your current status, which more often than not will be Neutral, which is how all City-state relationships start out. But you can get them angry by, for instance, moving a unit of your into their territory. If you only do it once, and give them time to get over it, they will go back to Neutral. You can also improve the relationship by pledging to protect them, but be careful. If they get attacked and you do not try to protect them, they will get very angry. Giving a gift opens a pop-up to say what kind of gift: 250 gold, 500 gold, 1000 gold, or a Unit. Note that a Unit is only worth 5 influence points, while 250 Gold is worth 20, so as I said Money is more powerful if your aim is to improve your status with them. So at Turn 242 I found the City-state of Singapore, and it appears that I was the first Empire to find them. So I immediately pledged to protect them. My Influence with them was 20, which is Neutral. We just met, and that is where things stand on first meeting. The be Friends you need to get to 30, and to be allies you need to get to 60. When I clicked Next Turn, I got a Quest from Singapore. They were worried about a Barbarian Encampment nearby, and if I cleared it out I would get additional influence with them. In this case, though, I let that go by. I want to move more quickly, and Singapore is across the Ocean from me. My influence with them at this point was 21, so only a modest increase. But I have 6038 Gold in my Treasury, and I am bringing in 102 per turn. So let’s see what a gift of 500 Gold will do. It brings me to 65 influence, so we are now Allies. But when I mouse-over Singapore, the pop-up window reminds me that my Influence will decrease by 1.12 per turn. For an Empire other than Austria this would mean a regular infusion of cash to keep up your status. And I have won Diplomatic victories with other Empires by saving up a lot of cash and dumping it on City-states just before the United Nations vote. But for Austria you have special ability called Diplomatic Marriage that lets you turn the City-state into a Puppet, and that is permanent. But it also presents some obstacles as we will see. We are allies now, but my influence will drop by 1 each turn, and I might lose my allyship before I can puppet them. But I can gift a unit and get another 5, and I happen to have a very obsolete Warrior unit that will serve the purpose. However, it takes three turns for the Unit to arrive, so I lose few more points. At Turn 247 I can now use the diplomatic Marriage option to make a puppet of Singapore. Prior to doing this I was fourth in score with 604, while the leader had 729. My Happiness Level was +25, and I now had 5833 Gold in my Treasury. Then I made Singapore a puppet, and now I am third in the game with a score of 664. My Treasury has fallen to 5258, which means it cost me 575 Gold. But most significant is that my Happiness fell from +25 to +9, which is a huge loss. I have enough gold to puppet 4-5 more City-states at this time but if I did I would have rebellions breaking out and my Empire would eventually collapse. This is the obstacle that Austria has to face. We need to promote Happiness before we go much further with making puppets. Civ is always a game of balances. Links https://civilization.fandom.com/wiki/City-state_(Civ5) https://civilization.fandom.com/wiki/Austrian_(Civ5) https://www.palain.com/gaming/civilization-v/playing-civilization-v-part-12/ Provide feedback on this episode.
-
-34
HPR4649: What did I do at work today? Part 3 Section 2
This show has been flagged as Explicit by the host. It is suggested reviewing the episode What did I do at work today? Part 3 Section 1 prior to listening Test driven development - a way of writing code that involves writing an automated unit-level test case that fails, then writing just enough code to make the test pass, then refactoring both the test code and the production code, then repeating with another new test case. - https://en.wikipedia.org/wiki/Test-driven_development Joplin - Joplin is an open source, cross platform note-taking app. - https://joplinapp.org/ PHP - A popular general-purpose scripting language that is especially suited to web development. Fast, flexible and pragmatic, PHP powers everything from your blog to the most popular websites in the world. - https://www.php.net/ MySQL - MySQL is an open-source relational database management system. MariaDB is a community developed fork of MySQL, often installing the MySQL package on a Linux distribution will actually install MariaDB. - https://en.wikipedia.org/wiki/MySQL - https://mariadb.org/ - https://www.mysql.com/ Sublime Text - Cross platform text editor - https://www.sublimetext.com/ Nmap - Network Mapper is a free and open source utility for network discovery and security auditing - https://nmap.org/ Markdown Fenced code blocks - "A code fence is a sequence of at least three consecutive backtick characters (`) or tildes (~). (Tildes and backticks cannot be mixed.) A fenced code block begins with a code fence, preceded by up to three spaces of indentation. The line with the opening code fence may optionally contain some text following the code fence; this is trimmed of leading and trailing spaces or tabs and called the info string. ... Although this spec doesn’t mandate any particular treatment of the info string, the first word is typically used to specify the language of the code block." ```ruby def foo(x) return 3 end ``` from CommonMark Spec at https://commonmark.org/ (CommonMark is a standard, interoperable and testable version of Markdown.) Writing to a Database with PHP The following PHP method is implemented within a database access class: function create_with_id($id, $name) { $born = time(); $id = mysqli_real_escape_string($this->db, $id); $name = mysqli_real_escape_string($this->db, $name); $sql = "INSERT INTO object (object_id, display_name_text, born, died) VALUES ($id, '$name', $born, 0); "; db_run_sql($this->db, $sql); } Note db_run_sql is a helper function defined elsewhere, not a built in function, and the property db is a previously initialized mysqli object. Provide feedback on this episode.
-
-35
HPR4648: Simple Podcasting - Episode 4 - Audio Analysis Fun
This show has been flagged as Clean by the host. 01 This is the fourth episode in a four part series on simple podcasting. 02 Introduction In this episode we will discuss alternatives to Audacity when it comes to analyzing audio spectrums to find the sources of unwanted noise. I previously promised some gratuitous hackery, and we will get into that in this episode. 03 Recall that with Audacity you first import the audio file, then select the part of the audio you wish to analyze (or ctrl-A for all), and then select analyze > plot spectrum. This is in fact the only feature of Audacity that I know how to use. I am definitely not an audio expert. I do however have some background in processing and analyzing other signals, so some of the basics are familiar to me. 04 We can accomplish the same thing that Audacity does in this instance provided we can do the following. First, we need to get the data out of the audio file and into a form which we can import into other software. Second, we need to perform certain mathematical operations on this data. Finally, we need to be able to plot the results of these calculations on a chart. -------------------- 05 Fourier Transforms First though, we need a bit of mathematical background. What Audacity is doing when it shows a plot of frequency versus amplitude is that it is showing the results of a Fourier Transform. A Fourier Transforms is a mathematical operation that converts the time domain into the frequency domain. Any complex signal, audio or otherwise, can be broken down into a collection of sine waves of various frequencies. For example, a simple square wave signal of say 100 hertz can be represented as a sine wave of frequency 100 hertz plus a collection of higher frequency sine waves which add together to give the sharp corners. 06 A Fourier Transform finds these sine waves and sorts them out into separate bins, with each bin representing an individual frequency or a collection of closely related frequencies, depending on how fine grained the sorting is. 07 This is exactly what we want when we are trying to figure out how to filter out noise. Recall that earlier in this series we had to solve a problem with a high pitched background noise which was originating in my cheap microphone. Analyzing this audio by frequency showed that it was a series of individual tones at 1 kHz intervals. We were then able to use filters targeted at those frequencies to get rid of that noise. 08 There are several optimized versions of the Fourier Transform algorithm. A very common one is the Fast Fourier Transform, common abbreviated to just "FFT". This is so common that the term "FFT" is often used to simply mean any Fourier Transform even though this is not technically correct. 09 Typical FFT algorithms require that the number of data samples is exactly a power of two. So the number of samples we need may be something like 4096, 8192, or 65536, to give a few random examples. When we transform from the time domain to the frequency domain, each sample becomes a single frequency "bin". So the more samples we have, the finer the resolution we get in terms of frequency. 10 If we assume we are dealing with flac files recorded at a 44.1 kHz sample rate, that is, 44100 samples per second, then if we have 32768 samples, each "bin" represents slightly more than 1 hertz. If we have 65536 samples, then each "bin" represents a fraction of a hertz. For our purposes we will pick 65536 samples. That means we need 1.48 seconds of data. For simplicity's sake we will record at least 2 seconds of data and then just discard the samples that we don't need. 11 There is a further complication here. Fourier Transforms normally work with complex numbers. Recall from your school days that as well as integers and real numbers there are complex numbers. Each complex number consists of two parts, a real component and an imaginary component. I won't go into the details of this, just accept that each sample needs to have two components. Fortunately, if we don't have complex number data we can just set the imaginary component to zero and use that. This is enough talking about the theory, let's get into the practical details. -------------------- 12 Extracting Data from Audio Files First we will look at how to extract the data from the audio files. Fortunately, one of the programs which we have already been using can do this. To do this we will use Sox. I am not aware of an equivalent feature in ffmpeg. 13 Sox calls itself "SoX - Sound eXchange, the Swiss Army knife of audio manipulation" Sox is free software and is licensed under the GPLV2 or later. In this case we want to use a feature which allows us to convert a binary audio signal file to a text data file. To convert the file to text data we just give the output file a ".dat" file extension and Sox will do this for us. 14 Here is a command example. sox inputfile.flac tdata.dat 15 This gives us a file in the following format, assuming this is a mono audio recording. ; Sample Rate 44100 ; Channels 1 0 0.045471191406 2.2675737e-05 0.055023193359 4.5351474e-05 0.048217773438 6.8027211e-05 0.053192138672 etc. The first line states the sample frequency The second line states that the data is for channel 1. The data starts on the third line. Column 1 is the time in seconds. Column 2 is the waveform data point. 16 To analyze the data we want a subset of these samples. When we convert from the time domain to the frequency domain, our resolution will be determined by the number of samples. We would like therefore to have at least as many samples as the sampling rate. We also want the samples size to be an even multiple of two. The number of points we want to have is equal to the next even multiple of two above our chosen sampling rate, 44,100 Hz. This number would be 65536. 17 To extract this data from the file we can do the following. tail tdata.dat -n+3 | head -n65536 | awk '{printf "%s\n", $2}' > tdata.csv 18 We use tail to skip over the first three lines. We use head to take the next 65536 lines and discard the rest. We use awk to extract the second column which we will use as the real component. We now have this data as a csv file in one column. -------------------- 19 Analyzing the Data To analyze the data we need software which can calculate FFTs. I will now show two examples of this, a very simple case using Libre Office Calc, and a more complex but more complete one using GNU Octave. 20 Using Libre Office We can do fourier analysis and plot charts using Libre Office. Take the csv file of data that we previously created. For this example I used data from a recording of silence so that I could see what internal noise was being generated by the headset. Open the csv file and import it into Libre Office Calc. 21 Now select all 65536 rows of column A. The Fourier function will automatically fill the imaginary component with zeros if we don't provide an column of imaginary numbers, so we don't need to provide a column of zeros. Then select Data > Statistics > Fourier Analysis. 22 A window will open allowing you to select various parameters. For Results to:, enter "D1". Grouped by Columns. Select OK. 23 New data should now appear starting in cell D1. The first line will say " Fourier Transform" The second line will state the input range. The third line will state "Real" in column D, and "Imaginary" in column E. The data will start in row 4. 24 For our simple example we will ignore the imaginary data and just use the real data, which will form our Y component when we plot it on a chart. We now need to create the X axis data. 25 Each cell is a "bin" of frequencies. Each cell therefore represents (sample frequency) / (Number of samples) Hz. 26 To create the X axis data showing frequency, enter the following formula in to column C to the left of each D column number. =((44100/65536) * (ROW() - 4) 27 We can now create an XY chart showing the frequency analysis. You may need to exclude the first couple of dozen rows as very low frequency components which cannot be heard may otherwise overwhelm the data we are interested in. Also, you only need the first half of the chart. The FFT mirrors the data from the first half of the array into the second half. 28 Because characterizing a sine wave requires a minimum of 2 points, although we have a sample frequency of 44.1 kHz, we really only have sound waves up to a maximum of half that, or 22.05 kHz. Create the chart with lines only. If you followed the above instructions, you should see something resembling what we saw in Audacity, except with each bin more sharply defined. 29 In the data that I had from a recording of unfiltered headset noise, I could see a distinct noise spike every 1000 hertz. 30 However, we have taken several shortcuts. First, the imaginary component of the data was ignored. Second, the magnitude (that is, Y axis) has both positive and negative peaks. Third, the data is not scaled to dB sound units, so we just have a relative measure. However, that by itself is enough to tell us where the frequencies are that we need to construct filters to deal with. 31 We could refine this spreadsheet a bit more to deal with the above issues, but I think we have demonstrated the basic principle, and working with a spreadsheet can be a bit awkward. However, if working with a spreadsheet is what you want to do, then you can add more columns and more formulae to improve on it. -------------------- 32 Other Analysis Software I will go on to GNU Octave in a moment, but I want to get a few other alternatives out of the way first. I won't go into any detail on them other than to point them out to people who want to have a go at trying these themselves. 33 Grace There is math and plotting software called Grace. This is free software, released under the GPL V2. According to the documentation, it seems to have the features we need, including an FFT function. However, I could not get it to work properly on Ubuntu 24.04. I could not get it to load a data file and plot data. 34 The error messages were vague and unhelpful. The file navigation system didn't work. There was no obvious path to success, and if it isn't easy to use then there is no point to it. This is fairly old software, designed for X Window and Motif. I gave up on it as not suitable for this series as I am looking for some fairly low effort things for people to try themselves. If someone else can get it to work on their PC, perhaps they could do an HPR episode on this themselves. 35 Command Line FFT Packages There are several command line FFT packages. They will read data from std in or from a file and output the FFT. However, these are not packaged for Ubuntu and appear to be distributed as C source code which you would download and compile. You can experiment with those if you wish, but I felt they were a bit out of scope for discussion here as I am looking at common tools that are ready to use. 36 Here are two examples. One is Command-line Fast Fourier Transform utility https://github.com/gregfjohnson/fft Another is cli-fft https://github.com/jonolafur/cli-fft 37 I have not tried these and cannot say whether they are any good or not. Similarly, there are a number of FFT packages that are libraries for languages such as Python. If you want to take the time to write a short program to go with them, you can create a dedicated FFT command line program. However, I felt that this too was out of scope for what I was trying to do here. 38 Doing it the Hard Way Hypothetically, it may be possible to write an FFT function in bash bc, which is the arbitrary precision calculator language which is part of the standard shell package. I say hypothetically, because I have not tried it. I think it would be an interesting challenge, but I don't have the time at the moment to try it. If anyone feels motivated to give it a try, they're welcome to give it a go and then do a podcast episode on it. -------------------- 39 GNU Octave We have seen that as well as using features built into Audacity to analyze the audio spectrum to see the frequencies of undesired noises, we were able to do the same using a Libre Office spreadsheet. 40 Now we'll look at another bit of software, GNU Octave. GNU Octave is free software, licensed under the GPL V3 or later. It is a mathematical scripting language, very similar to Matlab. People use it for mathematical, engineering, and scientific work. It can be found in most Linux distros and is available for some other operating systems as well. 41 Octave has two features built in that we need for our purposes. It does FFTs, and it has a plotting system built in to produce graphs. -------------------- 42 We will take the same audio test file that we used with Audacity and Libre Office and use it here as well. The bash script to convert the flac file to text data is essentially the same, with the exception that file extension on the output file as is ".txt" instead of ".csv". This latter change was an arbitrary decision on my part. 43 As a quick review, this bash script uses sox to convert a flac file to a text ".dat" file. Then it uses tail, head, and awk to extract the first 65536 rows of data, skipping over the header information and ignoring the first column of time data. This script will be in the show notes. -------------------- #!/bin/bash # This version is for use with the GNU Octave script. sox hsnoisemono.flac hsnoisemono.dat tail hsnoisemono.dat -n+3 | head -n65536 | awk '{printf "%s\n", $2}' > hsnoisemono.txt -------------------- 44 We now have a 1.1 MB file containing 65536 samples of data in text format. Now the next thing we need to do is to create a short Octave script file. I will just give a brief overview of the script here, the full script will be in the show notes. 45 I put the script in a file called "octavespectrum.m". I have never used Octave before now, but the convention seems to be to give the script a ".m" ending. The "she-bang" line is "#!/usr/bin/env octave". If you make the file executable you can run it like any other script, or you can type "octave" and then the name of the script to run. 46 I won't read out the script in detail, as that would be too hard to following along in a podcast. However, I pass several arguments to the script including the name of the data file, and then two integers that I use to limit the display area in the Y and X axes so I can have the chart focus on the areas of interest that I want to see. I also pass a string containing the name of the graphic file that I want the chart exported to. This was an arbitrary decision on my part and you can just hard code these values in if that is what you want to do. 47 The arguments are accessed by calling the "args()" function, which returns an array of strings. Next, it reads in the specified file using the "dlmread()" function. This reads all of the data into an array. 48 Next, it performs a hamming windowing function on the data. I'll explain that briefly. It is standard practice when doing FFT signal processing to "window" the signal. Since the signal sample is of finite length, it will stop at each end of the array. 49 Unless you were lucky enough for this to happen exactly at a zero crossing, this would produced an abrupt transition in the data which looks like "noise" to the FFT. The solution is to taper the signal off gradually towards the ends so that when it gets cut off the signal is fairly small at that point anyway. There are a variety of different windowing functions, but "hamming" seems to be the most commonly used. 50 Next, it does an FFT using the "fft()" function. 51 This gives us real and imaginary outputs. These are combined by summing the squares of each corresponding real and imaginary element and then taking the square root of each and storing that in a new array. This gives a single array of the same length as the originals, but combining the two output components. If anyone wants to tell me that this isn't how things are done in the audio world, they're welcome to make an HPR episode telling us all the right way to do things. 52 Then it does some scaling and selection of subsets of data so we get the X axis in hertz and just the number of samples that we wish to look at. If you are looking at the script, the thing to keep in mind is that Octave will work on entire arrays of data in a single operation. You don't need to write explicit loops for this. The looping is handled implicitly as part of the syntax. 53 It also does various other things that make the chart easier to read. The comments in the script describe these in more detail. Since this is a script it's easier to add these sorts of refinements than is the case for a spreadsheet so I have made the effort to add them. Finally it calls the "plot()" function. If an output graphics file name was provided, it also creates a PNG file containing the same image using the "saveas" function. 54 We now see the chart, and it looks more or less as expected. However, this chart is interactive. You can zoom and pan the data, something that you can't do with either Audacity or Libre Office. The chart window doesn't have a function for exporting the resulting chart to a "png" file, it will only save to an ".ofig" file. The ofig file is not a standard graphics file, it is a serialization of the chart data that can only be looked at using the Octave chart viewer. 55 Alternatively, you can just take a screenshot of the chart after you have interactively zoomed and panned to a point of interest. At the bottom left of the chart window is a pair of x-y coordinates which tell you the current position of the mouse pointer in chart units. This is very handy as it can be used to get the exact (or close to exact) frequency of each noise spike. 56 The Y axis is not scaled in any particular units such as dB, as I'm not sure how to do that according to audio industry conventions. On the other hand, I'm not sure that it's really necessary, as I don't know what dB means in tangible terms anyway. It does show relative sizes, so it helps to determine whether you have one noise frequency or multiple frequencies to worry about. 57 If anyone is familiar with how to scale the raw data from a flac file as exported by Sox into dB units according to audio industry convention, then they are welcome to create an HPR episode telling us how to do it. -------------------- 58 Comments on GNU Octave I had never used GNU Octave before this, although I had heard of it and it is quite a significant piece of software for a specific segment of users. 59 The syntax is a bit odd especially in how it deals with array operations, but I was able to google various examples and answers to eventually get this working. A few other peculiarities are that it uses the percent "%" character to denote a comment, and leaving out the semi-colon at the end of the line causes it to print the answer to the console after executing the statement. 60 The GNU Octave solution was harder to get working than the Libre Office method. However, once it was working it is easier to use repeatedly. If I were to want to automatically generate audio files with different filtering or other options and wanted to script the creation of a large number of images showing the results, this would be the way to do it. 61 When your run the Octave script you may get a warning which says something like "QSocketNotifier: Can only be used with threads started with QThread". This is apparently a routine warning message from the Qt graphics system which has no real significance in this context and can be ignored for our purposes. -------------------- 62 We now have a bash script which will use sox to extract the data from a flac file, and a GNU Octave script which can be used to display the resulting frequency spectrum. This does more or less the same thing as "Plot Spectrum" does in Audacity, but allows for zooming and panning to get a more detailed look at the data. 63 However it doesn't give you an absolute reading of the sound levels in dB, something that Audacity does provide. What I wanted it for though was to find the frequencies of the audible noise in the signal, something that it does quite well. -------------------- #!/usr/bin/env octave % Perform an FFT on the data in a file and plot the results. % ====================================================================== % The sampling frequency. This must be changed to accommodate the % actual sampling frequency if it was something else. samplefreq = 44100; % Thickness of line on plot. linewidth = 2; % ====================================================================== % The name of the data file is passed as a argument. args = argv(); if length(args) < 3 quit endif % File name. fname = args{1}; % Clip the peak values. peakclip = str2double(args{2}); % How much data to show, in kHz. rbound = str2double(args{3}) * 1000; % The optional file name to save a chart image to. if length(args) > 3 chartfile = args{4}; else chartfile = ""; endif % ====================================================================== % Read the data in from the file. sampledata = dlmread(fname); % Number of samples. samplecount = length(sampledata); % ====================================================================== % Window the data. This helps deal with the discontinuity of data at % each end of the array and the effects this has on introducing apparent % noise into the signal. windoweddata = (hamming(samplecount) .* sampledata); % ====================================================================== % Do the actual FFT. fftresults = fft(windoweddata); % Get real component. r = real(fftresults); % Get the imaginary component. i = imag(fftresults); % Combine the real and imaginary. In order to square each element of each % array, we must use the ".^" operator, not just "^". rfft = sqrt(r.^2 + i.^2); realfft = rfft(1:samplecount); % ====================================================================== % Scale factor for frequency. fscale = samplefreq / samplecount; % X axis scale, scaled to frequency. f = (0:samplefreq/2) * fscale; % Take a subset of the data if specified. rbound has to be re-scaled % from kHz to array increments. freq = f(1:min(rbound / fscale,length(f))); % y axis. We take the absolute value and then limit (clip) the peaks % so that a few large peaks don't obscure the smaller ones. mag = min(abs(realfft(1: length(freq))), peakclip); % Plot the results. figure; whandle = plot(freq, mag, 'LineWidth', linewidth); title(["Audio Spectrum of ", fname]); xlabel("Frequency (Hz)"); ylabel("Unscaled Magnitude"); grid on; % If the appropriate optional argument was specified, save the chart % to a file of that name. if length(chartfile) > 4 saveas(gcf, chartfile, "png"); endif % Need this so the plot window stays open. waitfor(whandle); % ====================================================================== -------------------- This is the shell script used with the above Octave script. The arguments are 1 - the file name for the input data file. 2 - The value to clip the peaks at. 3 - The upper frequency bound in kHz. 4 - The output graphics file name. #!/bin/bash octave octavespectrum.m hsnoisemono.txt 10 12 hsnoisemono.png -------------------- 64 Episode Conclusion In this episode we covered the following topics. What Fourier transforms are. Extracting data from audio files using Sox. Analyzing the data using Libre Office. Analyzing the data using GNU Octave. And, several alternative analysis methods. 65 Series Conclusion This is the end of a four part series on simple podcasting. In the first episode, we covered a simple podcast recording method. This first episode is all you really need to make a podcast. 66 In the second episode we covered basic filtering and a few other simple topics. The methods discussed in that episode provide basic improvements to your audio if you feel the need for it. 67 In the third episode we covered how to analyze audio noise problems using Audacity and additional filtering techniques to deal with specific problems that we may find. We also covered command line recording, playback, and getting information about an audio recording. 68 In the fourth episode we engaged in a bit of gratuitous hackery for the fun of it and showed how to use alternative software methods to analyze audio signals. 69 I hope that this series has been both useful and entertaining and that you will use the knowledge gained here to create and submit your own HPR podcast episodes. -------------------- -------------------- Provide feedback on this episode.
-
-36
HPR4647: UNIX Curio #7 - Compression
This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. In UNIX Curio #4 ( HPR episode 4617 ), I teased the subject of file compression. Today I'm circling back to that. The history of data compression goes back at least to the 1970s, and in contexts outside UNIX and computers, probably even earlier. Somehow, it is refreshing to learn that humans have always struggled to have enough storage space to keep all the data they want to hang on to. One way around this limitation is to use some form of compression. I am only going to dive into lossless compression for this episode—that is, a compression method that can be reversed and will spit out the original data bit for bit. Lossy compression methods also have their places: you might be familiar with their use for audio (such as Ogg Vorbis or MP3); it's also used for images (such as JPEG). Lossy compression allows some of the original data to be thrown away, resulting in a smaller file than is possible with lossless compression, but the intent is for the result to still sound or look "good enough" to a human observer. Also, I am going to limit my discussion to generic methods used for many types of data; while FLAC does lossless compression, it is specifically designed just for audio. I should make clear that I have never studied computer science or information theory, so this episode will not get into the science behind various types of compression algorithms and how they differ. But in general, these methods take advantage of the fact that many types of data have recurring patterns. English text mostly consists of words that often re-appear many times—source code similarly has keywords and variable names that recur. Compression is accomplished by representing a piece of data that occurs multiple times with a symbol that is shorter in length. The first compression program in the UNIX world I could find is called pack , from 1978 1 . It was shortly followed in 1979 by a similar program called compact 2 . Both of these used a technique called Huffman coding, but with some differences between them. Files compressed with pack were given a .z extension and compact gave filenames a .C extension. Roughly every five or ten years after this, a new program would come along and achieve lasting popularity. There were, and still are, two opposing forces facing any new form of compression. Working in favor was the advantages it provided—first among these was achieving a better compression ratio, but performance improvements such as speed or reduced memory usage could also be compelling. The force against any new method was the fact that it was not yet widely supported—it doesn't much help to have a smaller file if the people you share it with cannot decompress it. The next major advance in compression arose out of three scientific papers: two in 1977 and 1978 by Abraham Lempel and Jacob Ziv (called LZ77 and LZ78), and one by Terry Welch in 1984 which built on LZ78. This last method is typically referred to as LZW. Our UNIX Curio for today is a program called compress 3 that implements the LZW method. Files compressed this way are named with the extension .Z . I had always assumed that this was to honor Jacob Ziv, but now that I've researched the history, it seems more likely to be a follow-on from how files compressed by pack were named. Since pack did not use any of the Lempel-Ziv methods, I would guess that it used .z because that wasn't already taken by anything else, but that's pure speculation. I do recall encountering .Z files in the wild, but feel certain that hasn't happened in the last 25 years, maybe longer. If you need to expand one of these, uncompress 4 is the program to use ( GNU's gunzip can also handle them 5 ). However, there was a serious problem that arose with the LZ78 and LZW compression methods. Both of them were patented, and the owner became aggressive in seeking payment from developers and users. The compress utility was developed within two months of the publication of Welch's 1984 paper and was included in Bell Laboratories' Eighth Edition UNIX before these shakedowns started. The paper did not disclose that a patent had been filed, and apparently Spencer Thomas and the other developers of compress were unaware of it. The utility became popular for a while, and was even standardized by POSIX, but people moved away from LZW once the legal threats started. Another important advance came in 1991 and was called the DEFLATE compression method. It combined the un-patented LZ77 method with Huffman coding to achieve a similar level of compression as LZW (actually, often better) without the legal trouble. DEFLATE was developed for PKZIP and was soon adopted by the GNU project's gzip compressor. While Phil Katz (the "PK" in PKZIP ) patented one way of implementing the DEFLATE method, it was possible to write a compressor and decompressor without infringing 6 ; also, he apparently never tried to enforce the patent 7 . As I mentioned in UNIX Curio #4, .zip is both an archive and a compression format. Each archive member can be compressed with one of several possible methods (or stored without compression). Unlike a tar file where compression can be applied to the entire archive, in .zip each archive member is compressed individually. This often means a .zip file will be slightly bigger than a tar file with the same contents compressed with gzip , because the .zip format cannot take advantage of duplication that occurs among more than one member of the archive. The vast majority of .zip files use only the DEFLATE and uncompressed storage methods and these are the only options if you want to follow the profile standardized in ISO/IEC 21320-1. Actually, since they both use DEFLATE, gzip is able to extract a .zip file in the special case where it only holds one member compressed with that method. From the 1990s onward, people paid significant attention to avoiding patent landmines, so only methods that didn't have that problem became broadly popular. While the patents on LZ78 and LZW have since expired, I feel like their most successful legacy was in discouraging people from using those methods, leading to DEFLATE taking the popularity crown. The next step came in 1996 and 1997 with the development of bzip and bzip2 by Julian Seward. The original method was quickly followed by bzip2 , which was the version that achieved true popularity. They use the Burrows-Wheeler transform, which does not itself compress data but re-arranges it to make it more compressible; this is combined with other techniques 8 . (At least, that's my understanding. I told you, I'm not up on information theory.) This provides a significant reduction in the compressed size of the data compared to earlier methods—however, it is slower than DEFLATE both during compression and decompression. Separate projects have developed parallel versions of gzip and bzip2 that can take advantage of multi-processor machines, but the original utilities run single-threaded. Another five years later, in 2001, Igor Pavlov added the Lempel-Ziv-Markov chain algorithm (LZMA), an enhancement to LZ77, to his 7-Zip compression tool. This was followed a few years later by LZMA2, a container format that allowed for LZMA compression to be split between multiple threads. Broad LZMA2 support came to the UNIX world in 2009 with the xz utility 9 . It offers roughly similar compression ratios to bzip2 , though it can be better or worse depending on the data to be compressed. While compression generally takes even longer than bzip2 , decompression is significantly faster (though still not as fast as gzip ). The Linux kernel relatively quickly supported booting from xz-compressed images 10 because it was a good match for that use case—compression, the time-consuming activity, only has to be done once while the more frequent decompression during boot happens relatively fast. The last method I will cover is Zstandard 11 , often written as zstd . This came about in 2015, and is another variation on LZ77 that uses finite-state entropy (which means nothing to me, but you might understand it). It performs about as well as DEFLATE in terms of compression ratios, but is much faster both when compressing and decompressing data. I should say that these statements are true with the typical default settings—depending on the compression level selected, it can compress more slowly, but compress the data smaller. However, decompression is always speedier than DEFLATE. This makes it attractive for some uses, and it is heavily promoted by Meta/Facebook, where Yann Collet developed it. For example, shipping large amounts of actively-used data between machines in a data center can go more quickly when the size is reduced; however, if the compression and decompression steps take too long that benefit is lost. A speedy method can be valuable even if it doesn't result in the greatest reduction in size. This use case stands in contrast to, say, a compressed backup file which might only be accessed in a disaster recovery scenario or never accessed at all, making size more important than speed. Both the xz and zstd utilities have some built-in support for multi-threading, but the default is to run in a single thread. While xz can use multiple threads for decompression (but only if the file was compressed in multi-thread mode), the reference zstd utility can only use more than one thread for compression, not decompression. There are many other methods of lossless compression that have been developed over the decades, but I believe these are the ones you are most likely to encounter in the world of UNIX-like systems. This is a personal opinion, and others might choose a different set. As mentioned, it can be tough for a new method to gain popularity and 35-year-old DEFLATE is still probably the most commonly used despite not being the fastest or offering the greatest reduction in size. Even systems like FreeBSD, NetBSD, and OpenBSD that do not like to include GNU tools supported it by developing their own version of gzip based on the permissively-licensed zlib library. Technically, the LZW method used by the compress utility is still standardized by POSIX, so one might expect it to have the widest support. However, aggressive patent enforcement discouraged adoption, especially by Free and Open Source Software systems—even though the patent has expired, it is still out of favor compared to DEFLATE. For this reason, I feel justified in calling it a curio. References: Eighth Edition UNIX pack.c https://www.tuhs.org/cgi-bin/utree.pl?file=V8/usr/src/cmd/pack/pack.c 2.9BSD compact.c https://www.tuhs.org/cgi-bin/utree.pl?file=2.9BSD/usr/src/ucb/compact/compact.c Compress specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/compress.html Uncompress specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/uncompress.html GNU Gzip manual https://www.gnu.org/software/gzip/manual/gzip.html RFC 1951: DEFLATE Compressed Data Format Specification version 1.3 https://tools.ietf.org/html/rfc1951 History of Lossless Data Compression Algorithms: The Rise of Deflate https://ethw.org/History_of_Lossless_Data_Compression_Algorithms#The_Rise_of_Deflate bzip2 https://en.wikipedia.org/wiki/Bzip2 XZ Utils https://en.wikipedia.org/wiki/XZ_Utils 2.6.38 merge window part 2 https://lwn.net/Articles/423541/ zstd https://en.wikipedia.org/wiki/Zstd Appendix The table below demonstrates the results of compressing different types of data using tools described in this episode. While not totally rigorous, I did run each compression and decompression multiple times to ensure I was getting consistent results. The laptop I used has an Intel Core i5-6200U CPU running at 2.30GHz, and the system had at least 5 GB of free memory for each run. While this processor has two cores and can run four simultaneous threads, all utilities were run single-threaded. The term "best" means the highest level of compression available (the exact level used is shown). For bzip2 , the default is the best. For zstd , "best" is -19, which is the highest "normal" level, but "ultra" levels that are even higher also exist. Ratios are the percentage of the original size that the file was reduced to (other sources might instead express the compression ratio as the reduction in size achieved). In all results, smaller numbers are better. ┌────────────────────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐ │ │ gzip │ gzip │ bzip2 │ xz │ xz │ zstd │ zstd │ │ │(default -6) │ (best -9) │ (-9) │(default -6) │ (best -9) │(default -3) │ (best -19) │ ├──────────────┬─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 22,036,508 │ 21,891,623 │ 15,795,698 │ 13,487,768 │ 12,938,464 │ 20,454,657 │ 13,709,078 │ │ │ │ (24%) │ (24%) │ (17%) │ (15%) │ (14%) │ (23%) │ (15%) │ │English Text ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(90,532,092 │Compression │ 4.8s │ 7.6s │ 8.5s │ 49.8s │ 58.8s │ 0.6s │ 65.2s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 0.7s │ 0.8s │ 3.7s │ 1.2s │ 1.2s │ 0.4s │ 0.4s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 125,291,122 │ 124,189,544 │ 98,016,512 │ 84,882,492 │ 81,954,344 │ 120,604,855 │ 87,298,645 │ │ │ │ (21%) │ (21%) │ (17%) │ (14%) │ (14%) │ (20%) │ (15%) │ │Source Code ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(590,008,320 │Compression │ 22.0s │ 39.3s │ 54.8s │ 241s │ 298s │ 3.7s │ 348s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 5.1s │ 5.1s │ 20.3s │ 8.1s │ 7.8s │ 2.4s │ 2.4s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 32,830,905 │ 32,371,241 │ 26,856,579 │ 20,717,288 │ 20,352,880 │ 28,538,810 │ 23,154,582 │ │ │ │ (19%) │ (19%) │ (16%) │ (12%) │ (12%) │ (17%) │ (13%) │ │Binary Program├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(171,972,264 │Compression │ 6.4s │ 22.4s │ 18.6s │ 62.2s │ 67.8s │ 0.8s │ 111s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 1.5s │ 1.5s │ 5.6s │ 2.3s │ 2.3s │ 0.7s │ 0.7s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 146,397,772 │ 146,397,757 │ 144,485,451 │ 131,950,232 │ 130,926,780 │ 147,154,979 │ 145,703,840 │ │ │ │ (89%) │ (89%) │ (88%) │ (80%) │ (80%) │ (90%) │ (89%) │ │WAVE Audio ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(164,396,302 │Compression │ 9.2s │ 9.2s │ 25.1s │ 70.4s │ 97.7s │ 0.7s │ 58.3s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 2.0s │ 2.0s │ 13.5s │ 12.2s │ 12.1s │ 0.6s │ 0.8s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┴─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │ gzip │ gzip │ bzip2 │ xz │ xz │ zstd │ zstd │ │ │(default -6) │ (best -9) │ (-9) │(default -6) │ (best -9) │(default -3) │ (best -19) │ └────────────────────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘ English text consists of Titles 1 through 10 of the 2020 U.S. Code of Federal Regulations . Source code consists of a tar file containing the Linux kernel source, version 4.0. Binary program consists of an ELF-format executable of the pandoc application, version 2.17.1.1 found on Debian 12. Audio consists of a 24-bit Signed Integer PCM WAVE file with 2 channels at 44.1kHz, about 10:21 in length. For comparison, the audio-specific flac lossless compression utility reduced this file to 97,962,711 bytes (60%) in 2.6 seconds at the default (-5) level and to 97,714,876 bytes (59%) in 5.4 seconds at the highest (-8) level. Provide feedback on this episode.
-
-37
HPR4639: NLUUG Spring Conference 2026
This show has been flagged as Clean by the host. NLUUG Spring Conference 2026 "NLUUG is the association of (professional) Open Source and Open Standards users in the Netherlands" You can follow them on <a href="https://nluug.social/@nluug" rel="noopener noreferrer" target="_blank"> @[email protected] on Mastodon. I was particularly interested to attend their <a href="https://nluug.nl/evenementen/nluug/voorjaarsconferentie-2026/" rel="noopener noreferrer" target="_blank"> 2026 Spring Conference 2026 as our own <a href="https://hackerpublicradio.org/correspondents/0369.html" rel="noopener noreferrer" target="_blank"> Jeroen Baten was giving a talk on "Getting started with CI/CD using <a href="https://forgejo.org/" rel="noopener noreferrer" target="_blank"> Forgejo Actions and why this is important AF" He assures me he will post it as a show. cough owes me a show cough . While there the urge to record came upon me, so I was able to snag a few interviews. Ronny Lam representing NLUUG NLUUG is the association for (professional) developers, administrators and users of UNIX/Linux, Open Source, Open Source, Open Systems and Open Standards in the Netherlands. The NLUUG community includes, system administrators, programmers and network specialists. If you are working as an open professional, then NLUUG is the excellent association where you can keep track of your technical knowledge, for example during our six-monthly conferences. The aim of NLUUG is to disseminate the application and knowledge of open standards and UNIX/Linux. NLUUG maintains close ties with many organizations and individuals who pursue the open mind. <a href="https://nluug.nl/organisatie/personen/ronny-lam/" rel="noopener noreferrer" target="_blank"> https://nluug.nl/organisatie/personen/ronny-lam/ <a href="https://nl.wikipedia.org/wiki/NLUUG" rel="noopener noreferrer" target="_blank"> https://nl.wikipedia.org/wiki/NLUUG <a href="https://nluug.nl/" rel="noopener noreferrer" target="_blank"> https://nluug.nl/ Nico Rikken representing the FSFE The Free Software Foundation Europe is a charity that empowers users to control technology. Software is deeply involved in all aspects of our lives. Free Software gives everybody the rights to use, understand, adapt, and share software. These rights help support other fundamental rights like freedom of speech, freedom of press, and privacy. <a href="https://fsfe.org/freesoftware/index.en.html" rel="noopener noreferrer" target="_blank"> Learn more While we are no strangers to chatting with the <a href="https://fsfe.org/" rel="noopener noreferrer" target="_blank"> Free Software Foundation Europe (<a href="https://www.hackerpublicradio.org/eps/hpr0857/index.html" rel="noopener noreferrer" target="_blank"> hpr857 , <a href="https://www.hackerpublicradio.org/eps/hpr1957/index.html" rel="noopener noreferrer" target="_blank"> hpr1957 , <a href="https://www.hackerpublicradio.org/eps/hpr2223/index.html" rel="noopener noreferrer" target="_blank"> hpr2223 , <a href="https://www.hackerpublicradio.org/eps/hpr2945/index.html" rel="noopener noreferrer" target="_blank"> hpr2945 , <a href="https://www.hackerpublicradio.org/eps/hpr2946/index.html" rel="noopener noreferrer" target="_blank"> hpr2946 , <a href="https://www.hackerpublicradio.org/eps/hpr3388/index.html" rel="noopener noreferrer" target="_blank"> hpr3388 , <a href="https://www.hackerpublicradio.org/eps/hpr3407/index.html" rel="noopener noreferrer" target="_blank"> hpr3407 , <a href="https://www.hackerpublicradio.org/eps/hpr3833/index.html" rel="noopener noreferrer" target="_blank"> hpr3833 ), this was the first time we had a chance to interview <a href="https://nicorikken.eu/" rel="noopener noreferrer" target="_blank"> Nico Rikken . We chat about freedom and <a href="https://fsfe.org/activities/ada-zangemann/" rel="noopener noreferrer" target="_blank"> Ada and Zangemann - A Tale of Software, Skateboards, and Raspberry Ice Cream by <a href="https://k7r.eu" rel="noopener noreferrer" target="_blank"> Matthias Kirschner and <a href="http://sandra-brandstaetter.com/" rel="noopener noreferrer" target="_blank"> Sandra Brandstätter . Geert-Jan Meewisse representing Coalition for Fair Digital Education The Coalition for Fair Digital Education (CEDO) is a group of concerned parents, IT professionals, teachers, and privacy advocates committed to enabling fair and sovereign digital education. The coalition operates as a working group within Internet Society Netherlands (ISOC). We have drafted a manifesto calling for improvements in digital education. Today, children in education receive an online account from a foreign Big Tech company at an early age. Through this account, data can be collected, profiles can be built, and personal information can be used and exploited by these companies. This profiling leads to children being categorized and receiving tailored content that companies deem relevant—before they even discover things for themselves. And that’s not the only issue. Since schools exclusively use “standard” Big Tech solutions, children do not learn about alternative programs or tools. As a result, real digital skills and critical thinking are not developed, making children dependent on a company that profits from their data. The privacy and sovereignty of digital education are under severe pressure, affecting not only students but also teachers and parents, who are forced to use the same systems. Other countries are already ahead in this regard: in Denmark, Google products have been banned in schools in Helsingør municipality, and the German state of Baden-Württemberg has prohibited Microsoft 365. We advocate for the development of an open-source digital infrastructure for learning and educational tools, based on public values such as autonomy, equality, sovereignty, democracy, transparency, accessibility, academic freedom, and privacy-by-design. To achieve this, raising awareness among students, parents, teachers, and school boards is crucial. Additionally, we aim to involve policymakers by presenting our manifesto. <a href="https://eerlijkdigitaalonderwijs.nl/english/" rel="noopener noreferrer" target="_blank"> https://eerlijkdigitaalonderwijs.nl/english/ A working group of the <a href="https://www.internetsociety.org/" rel="noopener noreferrer" target="_blank"> Internet Society , Nico was here to tell us of their work to build a FLOSS alternative for Education. You can get in touch with him at gj -at- eerlijkdigitaalonderwijs .nl , or @geert-jan:matrix.org Conclusion I had great conversations with the sponsors who were a little shy about doing an interview. They do have a range of jobs available for those of us with Dutch nationality, and have lived in the Netherlands for the last 10 years. The event was fantastic, professional, held in a great venue, and the closest thing to real life <a href="https://xkcd.com/806/" rel="noopener noreferrer" target="_blank"> xkcd: Shibboleet as you are likely to get. I would like to thank the NLUUG team, volunteers, venue staff and of course the attendees for a wonderful day. With any luck this will not be the last time you hear about this team on HPR. The recordings will be available on the <a href="https://ftp.nluug.nl/video/nluug/2026-05-07-vj26/" rel="noopener noreferrer" target="_blank"> NLUUG FTP Server Provide feedback on this episode.
-
-38
HPR4638: Simple Podcasting - Episode 3 - Analyzing and Filtering
This show has been flagged as Clean by the host. 01 This is the third in a four part series on simple podcasting. 02 In this episode we will cover the following topics: Analysis of audio noise problems and filtering methods used to deal with specific problems that we may find. Command line recording. Command line playback. Getting information about an audio recording. 03 Introduction When I did my first couple of podcasts I didn't notice that there was a quiet high pitched whine or buzz in the background. Nobody complained about it, but I thought I could do better in subsequent episodes. 04 Creating an Audio Sample If you have a similar problem, the first step is to find out where it is coming from. If there is no audible noise where you are recording, there is a good chance the problem is in the microphone or another part of the audio system. Plug in your microphone and record 2 or 3 seconds of quiet audio where you do not speak into the microphone or make other noise. 05 You will need a minimum amount of data in order to analyze it. For a flac file sampled at 44.1 kHz, 2 to 3 seconds of data should be enough. To get a sample of just electronic noise you can put the microphone in a drawer or somewhere like that if you want to be sure of getting a quiet signal. Any sound recorded in this way should be mainly from the microphone or other electronic elements in the analogue pathway. To get a sample of possible ambient noise, such as fans, make sure the microphone is in the open air in an area which is representative of where it will be when you are recording. -------------------- 06 Analyzing using Fourier Transforms Next you need to look at the wave form. At this point I will describe this using Audacity. I will show other ways later, but Audacity is actually the easiest if you are starting from nothing. You don't need to become an expert in Audacity to use it, just follow the steps I will describe. I myself don't know how to use Audacity beyond using this one feature. 07 We are going to analyze the sound spectrum in our sample. The technique being used is a Fourier Transform. A Fourier transform, often called an "FFT" for fast fourier transform, is a mathematical method of showing a signal in terms of frequency along the x axis instead of time. This allows us to spot troublesome noise frequencies which appear when we don't want them to. The FFT is a very common mathematical technique which is widely used in signal processing, not just in audio. 08 There is software which will create pretty coloured animations of sound waves, but this is not what you want. These are simply decorative patterns and won't tell us what we want to know. -------------------- 09 Using Audacity Install Audacity if you haven't already. Start Audacity. Select file > import > audio, then navigate to your sample and select "open". The file should load. 10 In the wave form part of the window, click anywhere and then type Ctrl-S to select all data points. The chart should turn a slightly darker colour. From the menu, select Analyze > Plot Spectrum. A new window will open, showing magnitude in db on the Y axis, and frequency in hertz on the x axis. For "algorithm" be sure it is set to "spectrum" 11 There are now two settings that we need to play with while we look for problems. One is "size" The default for this is 1024. The other is "axis". The default for this is "log frequency". -------------------- 12 What to Look For What we are looking for are large obvious spikes that stand out in the data. Since our test signal has very little to no actual audio data, any spikes should represent electrical or other noise that doesn't belong there. 13 I have found two combinations of settings to be most helpful in finding problems. These are Size 2048, axis linear frequency. Size 32768, axis log frequency. 14 A small size value can help very narrow spikes stand out from the background more, while a large size value can help separate spikes from surrounding noise. A linear frequency axis can help with seeing all spikes across the full frequency range, while a log frequency axis can help to better see what is happening in the often very crowded lowest frequency range. -------------------- 15 A Real Example of an Audio Problem If you have good audio equipment you may find nothing obvious. If you cannot hear any noise in the signal, there may be none of any consequence and there is nothing for you to do. 16 However, in my case I found two main problems and one lesser one. One problem was a spike at 60 Hz, which is the AC line frequency. There is also a lesser problem of a collection of a broad frequency range of noise below 60Hz. Both of these however will be taken care of by the basic filtering that we looked at earlier so we do not need to worry about them here. 17 The other main problem is I had a large spike at every 1 kHz interval from 1 kHz to 19 KHz. This was noise generated within the head set electronics, or the result of noise on the USB power supply. This is the product of a cheap headset. 18 These spikes are not very large compared to the volume of my voice, but if I do the same sort of analysis of samples where I am speaking, they appear in the intervals between words. This results in a high pitched whine or buzz. This was the source of the background noise or buzz in my first two podcast episodes. I need to get rid of this. 19 One option would be to get a better microphone, but, well, that wouldn't be any fun would it. It would also cost money and I don't want to spend any of that if I don't have to. If you analyze your own signal, you may find a different pattern, or even no noise at all. If you did not find anything when shielding your microphone from ambient audio noise, repeat the same test but with the microphone exposed to acoustic noise in the room. -------------------- 20 Advanced Filtering The next step is to figure out how to get rid of this noise. I have called this section "advanced filtering", but we are actually just making use of a technique that was already covered in basic filtering. 21 To deal with the remaining spikes we can use additional "band reject" filters, each of which removes a specific frequency at 1 kHz intervals from 1 kHz to 12 kHz. We will use this in combination with the filtering that we have already done previously, so we don't need to worry about anything above 12 kHz as we already remove that with a low pass filter. After a small amount of experimenting I came up with the following. 22 Because I am applying a total of 16 filters, 4 for basic filtering and 12 to deal with the specific microphone problems that I have, I have broken up the filters into separate strings. I then generate the 12 new band reject filters from a template. Note that I don't show the "de-esser" filter here. I would recommend adding it as a separate step after doing the sort of filtering we are talking about here. 23 Rather than reading out multiple lines of bash script, I will post them in the show notes. I will give a brief description of them here which you can refer to when reading the show notes. The FFMPEG and Sox versions are very similar in concept so I don't need to go over the Sox version in detail. See the show notes for it. FFMPEG Version Here's the FFMPEG version. # The high and low pass filters. hlpfil="highpass=f=80, lowpass=f=12000" # Band reject filters filter for 60Hz and another for 50Hz. linefil="bandreject=f=60:width_type=h:w=20, bandreject=f=50:width_type=h:w=20" # Create a series of band reject filters, from 1 kHz to 12 kHz. # Change or remove this part if your recording hardware does not require it. ftemplate="bandreject=f=%s000:width_type=h:w=100" kilospikefil=$( seq 1 12 | xargs printf "$ftemplate," ) # Using ffmpeg ffmpeg -i input.flac -af "$hlpfil, $linefil, $kilospikefil" output.flac 24 There are a total of 5 lines of bash script. In the first line, we create a string called "hlpfil" which is just the high and low pass filters copied from our previous discussion on basic filtering. In the second line, we create a string called "linefil" which is just the simple bandreject filters to cover 50 and 60 hertz AC line noise filters also from basic filtering. 25 In the third and fourth lines, we create a string called "kilospikefil" containing the new filters. The "f" parameter represents the frequency we are targeting. The "w" parameter represents the "width" of the frequency range we are filtering in terms of hertz. The filter is applied gradually rather than with a sharp cut-off, so to get more filtering action we need to have larger width. In this case I decided to hammer the spike quite aggressively and so used a relatively wide width of 100 hertz. Testing with a voice file did not show any noticeable distortion, so it's an acceptable solution. 26 For this filter we need to create a dozen filter command so we use the shell "seq" command to generate a sequence of numbers from 1 to 12. We then pipe that into the xargs command which applies each number to the next command. The next command is "printf", which takes the number it gets from xargs and applies it to the "ftemplate" string template in a manner very similar to C programming printf string templates. 27 We also have a comma in there to separate each of the individual filters. We then surround this with a $ and () so we can run the command and capture the output into a variable. Then we call ffmpeg and pass it the filters we created by putting the variable names inside a double quoted string, separated by commas. All of this will be in the show notes, so don't worry about trying to get the exact details right now. Sox Version Here's the Sox version. # The high and low pass filters. sxhlpfil="highpass 80 lowpass 12000" # Band reject filters filter for 60Hz and another for 50Hz. sxfilter="$sxhlpfil $sxkilospikefil bandreject 60 20 bandreject 50 20" # Create a series of reject filters filters, from 1 kHz to 12 kHz. sxftemplate="bandreject %s000 100" sxkilospikefil=$( seq 1 12 | xargs printf "$sxftemplate " ) # Using SOX. sox input.flac output.flac $sxhlpfil $sxfilter $sxkilospikefil 28 The Sox version is very similar with the exception that the command arguments representing the filters must not be in quoted strings as Sox wants to see them as separate arguments instead of parsing a string. -------------------- 29 Confirming the Effect If we apply the above filters and look at this headset noise output file in the Audacity spectrum analyzer we will now see that these noise spikes are almost completely gone. We can now confirm how well this works by using a test audio file. Any normal short voice audio file will do for this. Just talk into the microphone normally and create a voice sample file that is 5 or 10 seconds long, or whatever you feel comfortable with. 30 With the original unfiltered voice audio I can hear a distinct high pitched whine overlaying the voice. With the filtered audio that whine or hum is not detectable. If we then look at the voice file in the Audacity spectrum analyzer, we can see distinct "notches" at the 50 Hz and 60 Hz frequencies, and at every 1 kHz from 1 kHz to 12 kHz. These notches are narrow enough that they won't cause a noticeable problem with voice signals. If we apply this filter to voice samples, the buzz or whine is gone and the voice signal sounds fine. Despite using a very cheap microphone, I now have acceptable quality audio for a podcast. 31 Again I want to emphasize that in this instance I am dealing with deficiencies with my hardware instead of buying a better microphone. These additional filters are intended to deal with the specific hardware problem I am facing. You don't need these additional filters if you cannot detect an audible problem. On the other hand, if you have a different problem you may wish to deal with a different set of frequencies. Finding these problems is the reason for using a spectrum analyzer. 32 FFMPEG has other filtering methods as well. However, as I didn't end up using them I can't really do an adequate job of describing them. If anyone has used them successfully, they are welcome to make a podcast on the subject. -------------------- 33 Completing the Process With these new filters added into the middle of the processing steps, you can now complete the processing by doing the de-essing, normalizing, and review steps as described in the previous episode. -------------------- 34 Command Line Recording I will now cover a separate topic, which is recording using command line programs. I am covering it in this episode as it is a short topic and it is convenient to talk about it here. 35 As well as using GUI based recording programs such as Gnome Sound Recorder, it is possible to record podcast episodes using command line tools such as FFMPEG. As for why you may wish to use command line tools to record audio, there are several reasons. One is that you may simply prefer to do it this way because it pleases you to do so. Another is that it allows the recording step to be included in a script that encompasses other parts of the process, automating what may have otherwise been separate manual steps. 36 However, if you don't find these arguments particularly compelling, then I'm not going to attempt to persuade you to use the command line to record audio. I am doing this part of this episode out of a desire to have a bit of fun and I probably won't be using it much myself. I will however use one of these methods to record this part of this episode. 37 Recording with FFMPEG - The Basics One of the common command line tools you can use is FFMPEG, a package which I have previously mentioned with respect to filtering audio files. Here is an example of how to record using FFMPEG. We call FFMPEG specifying the audio input system as the FFMPEG input, and then specify a file to output to. 38 # Record audio. ffmpeg -f pulse -i default ff.flac 39 Press 'q' to stop. This uses pulse audio on Linux for input "-f pulse", and the default input "-i default". However, this does not specify the the sample rate or mono recording. To do that we need to add a few more parameters as in the following 40 ffmpeg -f pulse -i default -ac 1 -ar 44100 ff.flac 41 "-ac 1" specifies mono output "-ar 44100" specifies 44.1 khz bit rate. 42 Playback with FFMPEG - The Basics FFMPEG can also play back music. In this case however we need to call the "ffplay" program rather than FFMPEG itself. To play an audio file, simply call ffplay and give it the name of the audio file as an argument to the command. For example: 43 # Play an audio file. ffplay podcast.flac 44 We can also call it with the "autoexit" option, which tells ffplay to automatically exit when the audio file has finished playing. ffplay -autoexit ff.flac 45 -autoexit means Exit when the audio file is done playing. 46 To exit in the middle of the recording, press "q' or ESC. To pause the playback, press "p" or space bar. To decrease the volume press "9" or "/". To increase the volume press "0" or "*". 47 To seek forward 10 seconds, press the right cursor button. To seek backward 10 seconds, press the left cursor button. To seek forward 1 minute, press the up cursor button. To seek backward 1 minute, press the down cursor button. 48 The "0" and "9" keys mentioned above are those on the top row of the keyboard, not the ones on the separate numeric pad. 49 While the recording is playing, a graphical window will open which shows a cascading waveform based on the current content. This is purely decorative and does not serve any particularly useful purpose. -------------------- #!/bin/bash # Record a podcast episode segment. # Get the next file name. # First we check if any matching file patterns exist. If they don't, # then we create the first one starting counting at 1. fcount=$( ls [0-9][0-9].flac 2>/dev/null | wc -l ) if (( $fcount < 1 )); then fname="01.flac" else # If there are any matching file patterns, we find the highest number # and increment it by 1. filenum=$( ls [0-9][0-9].flac 2>&1 | cut -d. -f1 | sort | tail -1 ) newfilecount=$(( 10#$filenum + 1 )) fname=$( printf "%02d.flac" $newfilecount ) fi echo "Recording to: $fname" # Record using ffmpeg. # This makes use of pulse audio and the input is the default audio input. # The sample rate is set to 44.1 kHz, and it is recorded as mono (1 channel). ffmpeg -f pulse -i default -ar 44100 -ac 1 $fname echo "Recorded audio to: $fname" # Report on basic information about the audio file that was just recorded. ffprobe -hide_banner $fname -------------------- 50 Sox - Not so Good I did not find the recording or playback features of Sox to be as useful as those of FFMPEG, so I won't bother to cover them here. -------------------- 51 Getting Information About an Audio Recording There are also command line tools which can be used to retrieve information about audio recordings. 52 FFMPEG Version With FFMPEG this is called "ffprobe". For example: 53 ffprobe hpr4566.mp3 54 This will print out a lot of information about FFMPEG itself. To skip that use the hide_banner option. 55 ffprobe -hide_banner hpr4566.mp3 56 This will print out information about the audio recording. This will include things like the duration, bit rate, sample rate, stereo or mono, etc. If the author added metadata tags to the file, it will also show those. HPR add things like the title, author, copyright license, comment, etc. You can extract the ones you want using something like grep and cut. 57 Sox Version Sox has a similar feature, called "soxi". 58 soxi ff.flac 59 However, it may not work on mp3 files if you do not have an mp3 handler for it installed. -------------------- 60 Conclusion In this episode we took a brief look at an example of how to solve an audio problem through filtering. We looked at how to use Audacity to find where the problems were. We then looked at how to apply filters to remove these sources of noise. We also looked at how to record podcasts and get information about audio files using command line tools. 61 In the next episode we will look at alternatives to Audacity for analyzing audio. While Audacity works just fine, this is an opportunity to have a bit fun with some gratuitous hackery. 62 This has been the third episode in a four part series on simple podcasting. -------------------- -------------------- Full Audio Processing Pipeline This version includes the special filters used to fix my headset problems. Use the version from the previous episode if you do not have the same audio hardware problems. #!/bin/bash # Full processing pipeline for making simple podcasts. # ====================================================================== # Concatenate multiple flac files into a single flac file. # This is used to combine podcast recorded segments into a single # flac file for uploading to HPR. concataudio () { outputname="$1" # First create the list file. printf "file '%s'\n" [0-9][0-9].flac > podseglist.txt # Now concatenate them ffmpeg -f concat -safe 0 -i podseglist.txt "$outputname" rm podseglist.txt } # ====================================================================== # Basic and advanced filters. filter () { inputfile=$1 outputname=$2 # Using ffmpeg. # The high and low pass filters. hlpfil="highpass=f=80, lowpass=f=12000" # Band reject filters filter for 60Hz and another for 50Hz. linefil="bandreject=f=60:width_type=h:w=20, bandreject=f=50:width_type=h:w=20" # Create a series of band reject filters, from 1 kHz to 11 kHz. ftemplate="bandreject=f=%s000:width_type=h:w=100" kilospikefil=$( seq 1 11 | xargs printf "$ftemplate," ) # Using ffmpeg ffmpeg -i $inputfile -af "$hlpfil, $linefil, $kilospikefil" $outputname } # ====================================================================== # De-Essing. deessing () { inputfile=$1 outputname=$2 option=$3 # De-essing filter. ffmpeg -i $inputfile -filter_complex "deesser=i=0.5:m=0.5:f=0.5:s=$option" -b:a 336k -sample_fmt s16 $outputname } # ====================================================================== # Normalizing the audio to EBU R128 standard for review using ffmpeg. normffmpeg () { inputfile=$1 outputname=$2 # Normalize to EBU R128 standard. ffmpeg -i $inputfile -af loudnorm=I=-17:TP=-2.0:LRA=4.0 -ar 44.1k $outputname } # ====================================================================== # Output an MP3 version to help with reviewing. mp3convert () { inputfile=$1 # Get the name of the file and then create the output file name. j=$( basename $inputfile ".flac" ) outputname="$j"".mp3" # Convert to MP3. ffmpeg -i $inputfile $outputname } # ====================================================================== # Concatenate the separate audio files. concataudio fullpod-unfiltered.flac # Basic filtering. filter fullpod-unfiltered.flac filtered.flac # De-essing. This is the version to send for publishing. # The third argument should be "o" for de-essing, or "i" for pass through without de-essing. deessing filtered.flac fullpod.flac o # Normalized for review. normffmpeg fullpod.flac fullpod-norm.flac # Output an MP3 copy for review. mp3convert fullpod-norm.flac -------------------- -------------------- Provide feedback on this episode.
-
-39
HPR4637: UNIX Curio #6 - at and batch
This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. I would imagine that most users of UNIX-like systems have heard of cron —certainly any system administrator should have. Briefly, cron is a way of running a job repeatedly based on the time and date; for example, a job could run every hour, at 5:00am every Tuesday, or the 3rd of every month. It is commonly used for administrative or maintenance tasks that should be done on a regular schedule, such as checking for software updates, rotating log files, or updating the database for the locate command. As well-known as cron is, there is a similar utility that very few seem to be aware of: at . This is the word "at", and has nothing to do with the at symbol "@". An at job is very much like a cron job, except that an at job only runs one time. A job is submitted by running at timespec 1 , where timespec is the time and date the job is to be run. The linked POSIX specification page describes acceptable formats for timespec ; some examples are " now ", " 14:00 ", " noon tomorrow ", " 14:00 + 3 months ", and " 14:00 January 19, 2038 ". The utility then waits on standard input for you to enter a set of commands to be run in the job. You end input by typing Control-D to mark the end of text. (As an alternative to typing in the job, you could instead use the "<" symbol to redirect standard input to come from a file containing the commands you want to run.) When the specified time arrives, the job will be run. That is the theory, anyway, but some things may interfere. The normal configuration for some implementations only checks for due at jobs every five minutes, so there can be a delay before a job is actually run. Also, if the system isn't running, obviously it can't execute any jobs. When it comes back up, typically it will check for any pending at jobs that are currently or past due and run those. It is best to think about an at job being run no earlier than the time it was scheduled for, and probably soon after, provided the system is up. The POSIX standard doesn't specify anything about when jobs are actually run, just that they are scheduled for a particular date and time. The user does not need to be logged in for a job to run—if the job outputs anything to standard output or standard error, that text will be e-mailed to the user, presuming the system is set up to send mail. This is often true for a server, which might be running a Mail Transfer Agent like sendmail , postfix , or exim , but many desktops are not. If nothing is output to standard output or standard error, or if that output is redirected to a file, then mail will not be sent on job completion. This behavior can be changed with the -m option; in that case, mail will always be sent when the job finishes whether or not there is any output. The batch command is very similar 2 —POSIX specifies it as being equivalent to at now with two differences. First, jobs are put into a different queue, and second, mail is always sent when a job completes as if the -m option was used with at . In practice, however, certain aspects of the behavior of batch depend on the implementation. On the large majority of systems I investigated 3,4,5,6,7,8 , but not all 9 , batch jobs will only be run when the system load level drops below a certain point. This can typically be configured by the administrator but has a default value—the manual pages for a couple systems don't actually list a default value and just say batch jobs will run "when system load levels permit". Basing execution on the load level makes sense if the batch utility is seen as a way of running potentially resource-intensive jobs when the system is not being heavily used. However, this behavior is not required by POSIX. Another question that the standard leaves unanswered is how queues behave. From the normal understanding of the word "queue", you might expect that each successive job is run one at a time once the previous job completes. However, this is not stated in POSIX, and some implementations explicitly allow a configurable number of jobs to run simultaneously. Manual pages for other systems simply don't mention the subject. (I researched this episode by looking at documentation for a number of BSD, Linux, and commercial UNIX systems, but didn't actually test out how they behave.) POSIX only requires systems to have two queues, one named "a" for at jobs and one named "b" for batch jobs, but allows implementations to have more. It says nothing about how different queues compete for resources—one implementation assigns a higher nice value to jobs in a queue whose name comes later in the alphabet, giving them a lower priority in the process scheduler. So what good are at and batch ? While I think they certainly meet the "obscure" requirement for a UNIX Curio, I have to admit they aren't particularly useful today. They were designed for an era where a typical UNIX-like system would run around the clock and had multiple users who might log in at various times of the day but weren't connected 24/7. In that context, using batch to run a job when the system is lightly loaded might be useful; nowadays, you can just run it whenever you like on your own machine. I have never actually used batch myself. On a machine where there is serious competition for resources among users, batch is probably not a sophisticated enough tool to manage their jobs—the NetBSD and Debian manual pages explicitly suggest using something different 3,6 . Supercomputing environments have even more complex requirements and a number of specialized solutions exist for scheduling jobs there. I have used at a couple of times. One example was for an organization I was part of that had paid for its domain name registration several years into the future. On the organization's server, I set an at job to e-mail the administrator a reminder to renew it a few months before the domain was due to expire. It was useful in that case because I didn't know whether I would even continue to be involved then, so a personal reminder for myself wouldn't necessarily help. But in my experience, administrative tasks don't tend to be one-off events. Instead, they repeat, making cron the right tool to use. For reminders, a calendar app is probably a better solution in most cases. While you might never have a use for at and batch , I still think it's good to know that they exist. Just be aware that you'll probably need to read the manual page on your system to fully understand how they will behave. References: At specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/at.html Batch specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/batch.html NetBSD 10.0 at manual page https://man.netbsd.org/NetBSD-10.0/at.1 FreeBSD 15.0 at manual page https://man.freebsd.org/cgi/man.cgi?query=at&sektion=1&manpath=FreeBSD+15.0-RELEASE+and+Ports OpenBSD 7.8 at manual page https://man.openbsd.org/OpenBSD-7.8/at.1 Debian 13 at manual page https://manpages.debian.org/trixie/at/at.1.en.html openSUSE 42.3 at manual page https://man.freebsd.org/cgi/man.cgi?query=at&sektion=1&manpath=openSUSE+42.3 HP-UX Reference (11i v3 07/02) - 1 User Commands A-M (vol 1) https://support.hpe.com/hpesc/public/docDisplay?docId=c01922490&docLocale=en_US OpenSolaris 2010.03 at manual page https://man.freebsd.org/cgi/man.cgi?query=at&sektion=1&manpath=OpenSolaris+2010.03 Apologies for the "tapping" sound that occurs in parts of this episode. I think my microphone must have picked up some electromagnetic interference. Provide feedback on this episode.
We're indexing this podcast's transcripts for the first time — this can take a minute or two. We'll show results as soon as they're ready.
No matches for "" in this podcast's transcripts.
No topics indexed yet for this podcast.
Loading reviews...
ABOUT THIS SHOW
Hacker Public Radio is an podcast that releases shows every weekday Monday through Friday. Our shows are produced by the community (you) and can be on any topic that are of interest to hackers and hobbyists.
HOSTED BY
Hacker Public Radio
CATEGORIES
Loading similar podcasts...