EPISODE · Apr 18, 2024 · 50 MIN
"Saga of a Gnarly Report" with Owen and Dan
from Elixir Wizards · host SmartLogic LLC
In today's episode, Elixir Wizards Owen and Dan delve into the complexities of building advanced reporting features within software applications. They share personal insights and challenges encountered while developing reporting solutions for user-generated data, leveraging both Elixir/Phoenix and Ruby on Rails. The discussion zeroes in on crucial data modeling and architectural decisions that enhance reporting efficiency and flexibility. Owen and Dan explore tactics like materialized views, event sourcing, and database triggers to optimize data handling while being mindful of UX elements like progress indicators and background job management. They share insights on leveraging the Elixir/Beam ecosystem’s strengths—like concurrency and streamlined deployment—to tackle common reporting, caching, and integration challenges. The episode highlights the impact of reporting features across all aspects of a software application’s design and architecture. Key topics discussed in this episode: Reporting on assessment data, survey results, and user metrics Differences between reporting and performance/error monitoring Implementing reporting in Elixir/Phoenix vs. Ruby on Rails Displaying reports in web, printable, PDF, SVG, and CSV formats Challenges of generating PDFs for large data sets Streaming CSV data directly to the client Handling long-running report generation tasks Providing progress indicators and user notifications Strategies for canceling or abandoning incomplete reports Tradeoffs of pre-calculating report data vs. real-time generation Materializing views and denormalizing data for reporting Exploring event sourcing patterns for reporting needs Using database triggers and stored procedures for reporting Balancing data structure optimization for reports vs. day-to-day usage Caching report data for faster retrieval and rendering Charting and visualization integration in reporting systems Links mentioned: Prometheus monitoring system & time series database https://prometheus.io/ Thinking Elixir "FLAME with Chris McCord" https://podcast.thinkingelixir.com/181 Phoenix LiveView Uploads https://hexdocs.pm/phoenix/file_uploads.html https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.UploadWriter.html Postgrex PostgreSQL driver for Elixir https://hexdocs.pm/postgrex/Postgrex.html Ecto https://hexdocs.pm/ecto/Ecto.html Heroku cloud application platform https://www.heroku.com/ Elixir Wizards S9E12 Marcelo Dominguez on Command and Query Responsibility Segregation https://smartlogic.io/podcast/elixir-wizards/s9-e12-marcelo-dominguez-cqrs/ Commanded Elixir CQRS/ES applications https://github.com/commanded/commanded Tailwind CSS Framework https://github.com/tailwindlabs Memcached https://memcached.org/ Redis https://redis.io/ Oban https://hexdocs.pm/oban/Oban.html ETS https://hexdocs.pm/ets/ETS.html Capistrano remote server automation and deployment tool https://capistranorb.com/
What this episode covers
In today's episode, Elixir Wizards Owen and Dan delve into the complexities of building advanced reporting features within software applications. They share personal insights and challenges encountered while developing reporting solutions for user-generated data, leveraging both Elixir/Phoenix and Ruby on Rails. The discussion zeroes in on crucial data modeling and architectural decisions that enhance reporting efficiency and flexibility. Owen and Dan explore tactics like materialized views, event sourcing, and database triggers to optimize data handling while being mindful of UX elements like progress indicators and background job management. They share insights on leveraging the Elixir/Beam ecosystem’s strengths—like concurrency and streamlined deployment—to tackle common reporting, caching, and integration challenges. The episode highlights the impact of reporting features across all aspects of a software application’s design and architecture. Key topics discussed in this episode: Reporting on assessment data, survey results, and user metrics Differences between reporting and performance/error monitoring Implementing reporting in Elixir/Phoenix vs. Ruby on Rails Displaying reports in web, printable, PDF, SVG, and CSV formats Challenges of generating PDFs for large data sets Streaming CSV data directly to the client Handling long-running report generation tasks Providing progress indicators and user notifications Strategies for canceling or abandoning incomplete reports Tradeoffs of pre-calculating report data vs. real-time generation Materializing views and denormalizing data for reporting Exploring event sourcing patterns for reporting needs Using database triggers and stored procedures for reporting Balancing data structure optimization for reports vs. day-to-day usage Caching report data for faster retrieval and rendering Charting and visualization integration in reporting systems Links mentioned: Prometheus monitoring system & time series database https://prometheus.io/ Thinking Elixir "FLAME with Chris McCord" https://podcast.thinkingelixir.com/181 Phoenix LiveView Uploads https://hexdocs.pm/phoenix/file_uploads.html https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.UploadWriter.html Postgrex PostgreSQL driver for Elixir https://hexdocs.pm/postgrex/Postgrex.html Ecto https://hexdocs.pm/ecto/Ecto.html Heroku cloud application platform https://www.heroku.com/ Elixir Wizards S9E12 Marcelo Dominguez on Command and Query Responsibility Segregation https://smartlogic.io/podcast/elixir-wizards/s9-e12-marcelo-dominguez-cqrs/ Commanded Elixir CQRS/ES applications https://github.com/commanded/commanded Tailwind CSS Framework https://github.com/tailwindlabs Memcached https://memcached.org/ Redis https://redis.io/ Oban https://hexdocs.pm/oban/Oban.html ETS https://hexdocs.pm/ets/ETS.html Capistrano remote server automation and deployment tool https://capistranorb.com/
NOW PLAYING
"Saga of a Gnarly Report" with Owen and Dan
No transcript for this episode yet