Rocket Lawyer’s Tim Silverline on Why Clean Pentest Reports Can Be Red Flags

When Tim Silverline received a pentest report that was essentially a clean bill of health with zero evidence of actual testing, he knew his security program had a problem. As Vice President of Security at Rocket Lawyer, this experience sparked a complete transformation from annual security theater to continuous, evidence-based testing that provides actionable intelligence — with Sprocket! In his chat with Casey, recorded at RSA 2025, Tim shares hard-earned insights about building effective security programs in established organizations while navigating the complexities of rapid AI development and multi-compliance requirements.    Tim touches on how static analysis tools create more noise than value, explaining how packages flagged as critical vulnerabilities often aren't even loaded into memory or used in exploitable ways. His solution involves runtime analysis with eBPF sensors that monitor actual execution rather than theoretical package inventories. He also discusses the unique challenges of implementing SOC 2 controls in an 18-year-old company versus a startup, emphasizing the critical importance of executive alignment before attempting cultural transformation.    Topics discussed: The limitations of traditional annual penetration testing and why continuous testing provides better coverage for organizations with rapid deployment cycles. How runtime analysis with eBPF sensors eliminates false positives by monitoring actual code execution rather than static package inventories that generate noise. The strategic approach to managing SOC 2 compliance implementation in established organizations, focusing on executive alignment before attempting cultural transformation. Advanced attack surface management techniques that extend beyond hosted applications to include third-party platforms and exposed API keys. The challenge of staying ahead of AI development from a security perspective, particularly as interconnected AI models create complex data flow patterns difficult to audit. Why clean penetration test reports with no evidence of actual testing indicate vendor problems rather than strong security posture. The evolution from static vulnerability scanning to context-aware prioritization based on actual exploitability and system exposure. Strategies for integrating security findings into development workflows through two-way JIRA integration and regular cross-team security reviews. The growing complexity of non-human identity management as DevOps practices increase the proliferation of API keys and service accounts across cloud environments. How the NextJS vulnerability response demonstrates the value of runtime monitoring for rapidly identifying which instances actually use vulnerable middleware configurations. Listen to more episodes:  Apple  Spotify  YouTube Website