R/Pharma Summit

About

REGN Attendees

  • Peter Ehmann (PMQTS)
  • Nick Giangreco (PMQTS)
  • Ryan Yu (BDM)
  • Sri Pavan Vemuri (BDM)

Atorus

  • Steve Nicholas
  • Ashley Tarasiewicz

ROI in Open Source

Why use open-source?

  • collaboration, faster innovation
  • not just R, but python
  • opens up talent pool
  • not spending time training to learn SAS
  • what are students coding in grad school

What is the ROI?

  • difficult to measure
  • prove it with POC
  • tipping point (financial cost)
    • propriety vs.
    • open-source
  • experiment SAS vs. R timing to program SDTM tables
    • automation
    • customization
    • final product
  • not just cost, but mindset shift
  • PMQTS - analagous to shift from siloed coding to best practices
    • savings are not apparent up-front
    • ROI is difficult to calculate and communicate
    • POC and reach tipping point

Who do we need to influence?

  • health authorities
  • business IT

What to demystify?

  • validation
  • reproducibility
  • open-source

Quality in Open-Source

What is quality

Quality Assurance (QA) and Quality Control (QC) are two distinct but interconnected components crucial to ensuring pharmaceutical products’ safety, efficacy, and reliability. QA and QC in the pharmaceutical industry collectively form a comprehensive framework for maintaining high-quality standards throughout pharmaceutical manufacturing.

Benefits of open-source

  • TFL generation
  • Multi use tools: shiny, packages (don’t have to start from skratch)
  • Data repository

Why open-source

  • User facing vs. Quality facing documentation (know your audience)
  • User -> see underlying code and flag issues
  • Quality -> should know author(s) and tests performed (maintenance strategy)
  • Maintenance
    • project re-assinged
    • employee leaves (turnover)
    • phase out
    • GIT!!! - learning curve is steep (barrier to entry)
      • development
      • transparency
      • branching - try things and not break
    • standardization vs. harmonization !!!
    • consider yourself as a collaborator
    • intangible
      • investment
      • eliminating resistance to change
      • embrace new approaches
      • speak the same language (get on the same level) - use non-technical terms
      • “thats how it worked before” is not a reason (ex Gen AI)
      • rigid vs flexible mindset

GenAI and Clinical Trials

  • Gilead
  • Eli Lilly

LLMs

  • LLM is an instruction following engine with no memory
  • Provide:
    • precise instructions
    • supply the right information at the right time
  • Tool Calls
  • Model, Context, Protocol (MCP)
  • evals are critical for testing and development
  • we can’t test everything, and LLMs will screw up
    • adopt version control (oversight)
  • create ways to find errors
  • Current state: do not make plans before piloting
    • play with current tools
    • do not plan to put things in production in 2025
    • figure out what it is good at, vs not good at
  • Use cases:
    • mostly documentation
    • nobody mentioned as junior analyst (@Natalie)
  • Build foundation to be ready when the models improve
  • Is it doing what it’s meant to do?
    • Give it to high performers
    • If it fails, it will get shut down early.

Comprehensive C-QT Analysis From Data to Deliverables

https://a2-ai.github.io/cqtkit-docs/versions/1.0.2/

GenAI in Positron

  • Polyglot IDE - R, Python, SQL, Rust, Julia,
  • Fork of VS Code
  • Claude integration into Positron
    • knows all environment variables

Validation Hub

  • R consortium project
  • 10 organizations
  • Involvement from health authorities

Roundtable 1 - Perspectives of CROs

  • Denali
  • Cytel
  • Atorus
  • Syneos

Takeaway: Need to have time for learning & leveling up to enable open-source adoption.

Roundtable 2 - Open-Source Change Manangement

  • GSK
  • Genentech/Roche

Takeaway: Trainings are ineffective, need to implement best practices in everyday work.

  • GSK uses “just in time training”
  • Advanced programmer is teamed with novice programmer to implement best practices in a real-world setting
  • Coders will only adopt change if they have to