EYE on NPI - Analog Devices Inc MAXREFDES178 and MAX78000

This week's EYE ON NPI is a cool cubic camera - it's the Analog Devices Inc. (nee Maxim Integrated) MAXREFDES178 (https://www.digikey.com/short/t4052tpj), an all-in-one vision eval board for the MAX78000 (https://www.digikey.com/en/products/detail/analog-devices-inc-maxim-integrated/MAX78000EXG/13155097). The MAX78000 is a specifically customized ARM Cortex M4 microcontroller that has a Convolution Neural Network sub-circuit. This means that it's really good at doing the kind of mathematical calculations required to do vision or audio-based machine learning inferences without the need for a heavy-duty processor like in a Single Board Computer or Internet co-processing. This makes it great for stand-alone smart camera applications - which is why the MAXREFDES178 (https://www.digikey.com/short/t4052tpj) is an excellent eval board that might even be able to be integrated in your final design. The MAXREFDES178 camera kit (https://www.digikey.com/short/t4052tpj) comes as a small hackable cube with two MAX78000s - one for video and one for audio processing. A 1.5" 240x240 resolution TFT with capacitive touchscreen, OVM7692 camera module (https://www.digikey.com/en/products/detail/omnivision-technologies-inc/OVM7692-RAAA/3190565), I2S Codec, I2S microphone, 5 buttons, Lipoly Battery, USB Type C with battery charging, and microSD card slot that can be used to load 'Apps' onto the processors. There's also a cute SWD programmer dongle that plugs into the USB so you can do full programming and step debugging of your own application. The strength of the MAXREFDES is that it demonstrates the audio and video inferencing that normally would take a Raspberry Pi-like SBC computer or an online connection for the heavy computation. But, since it's done on-chip, the power draw is much much less and the startup time is nearly instant. This makes it great for portable, battery, or even solar / power-harvesting applications (https://www.youtube.com/watch?v=TvuLv13EGlM) As with most AI products, the biggest challenge for starting out is how to train the model that will be used for inferencing. The MAXREFDES178 comes with a basic voice-detection model that covers about 20 words, or about 10 celebrity faces. However, unless you happen to be designing a paparazzi product you'll need to create the model that detects exactly what you want and then load that into the MAX78000's CNN memory. Analog Devices has a video series (https://www.analog.com/en/education/education-library/videos/video-series/understanding-artificial-intelligence.html) and GitHub documentation repo (https://github.com/MaximIntegratedAI/MaximAI_Documentation) that goes through the process of how to do the training: you can use the popular PyTorch or TensorFlow Lite environments. Note that since the CNN is in hardware, not 'bitbanged' on the main core, you do not have to use TF Lite for Microcontrollers (https://www.tensorflow.org/lite/microcontrollers) - although you could probably run the TFLiteMicro codebase if you wanted, on the MAX78000, it just wouldn't get any hardware acceleration. Creating the model will require collecting a ton of data, image or voice, and then training using a CUDA-powered desktop or cloud computer. If you're interested in edge AI development (https://www.analog.com/en/thought-leadership/the-dream-of-edge-ai.html), the MAX78000 is an excellent chip to use with a familiar ARM M4 core and tons of peripherals to support any external sensors or devices needed - no co-processor needed! There's even a Feather board available (https://www.digikey.com/en/products/detail/analog-devices-inc-maxim-integrated/MAX78000FTHR/13549063), but for quick audio/video development, the MAXREFDES178 camera kit (https://www.digikey.com/short/t4052tpj) works right out of the box and it's in stock right now at DigiKey for immediate shipment! Order today and it'll ship immediately so you'll be convolving and neuralling by tomorrow afternoon. See more in the video here https://www.youtube.com/watch?v=7Toaz-0vdIE