mlnewton
Writing Work Resume

Krea 2Matthew Newton

Krea 2 Krea 2
Krea 3 Krea 2 Other Research Drip.art Just for Fun

Selected Work

Krea 3

Krea AI · Image editing

Krea 3 placed in the top five in preliminary Artificial Analysis image-editing results and is expected to rank highly on Design Arena (October 2026; public announcements pending).

More training contributions

Contributed across Krea 3's full stack: pretraining, midtraining, post-training, prompt expansion, and the inference harness.

The scoring and captioning labels came from open-source VLMs, not proprietary-model distillation.

  • Built embedding-search infrastructure to efficiently index our image collection and mine editing pairs, using open-source VLMs to generate scores and captions.
  • Worked on Fast and Ultra prompt expanders and inference harnesses, and on pretraining and post-training evaluations and ablations.
  • Owned data curation and pipelines, using midtraining to close concept-coverage gaps, from entities to camera angles and styles.
  • Helped train VLM-based prompt expanders through SFT and RL, contributing text rewards and safety training.
  • Led collection of human annotation data for DPO and reward model training.

Krea 3 announcement ↗

Krea 2

Krea AI · Text-to-image generation

I worked across Krea 2's development on a small team, spanning pretraining, midtraining, post-training, and prompt expansion. My data work included curation and sampling pipelines to broaden concept coverage, and leading human annotation data collection for DPO and reward model training. The examples below highlight a few pieces of that work, from world knowledge to moodboard-driven generation.

Krea 2 reached the global top 10 on Artificial Analysis's text-to-image leaderboard ↗ (June 2026). More than 16 million cumulative Hugging Face downloads as of October 17, 2026, across its Raw ↗, Turbo ↗, and Comfy-Org ↗ repositories.

A few training contributions
  • Ran pretraining and post-training evaluations and ablations with teammates to find what actually helped.
  • Built the prompt expander from SFT through RL, designing text rewards to complement the team's image rewards.
    • Built moodboard-driven prompt expansion.Turned moodboards into text conditioning, selecting reference images for texture and style.
    • Added safety training to reduce unintended IP leakage and unwanted nudity while preserving accurate anatomy and artistic expression.
  • Owned data curation and midtraining concept coverage.Owned data curation and pipelines, using midtraining to close concept-coverage gaps, from entities to camera angles and styles.
  • Led collection of human annotation data for DPO and reward model training.

World Knowledge and Concept Coverage

I built the data pipelines to keep the long tail in the training mix. Embedding-based clustering broadened visual coverage; Wikipedia/Wikidata-guided sampling and caption searches helped retain named entities that ordinary cluster sampling could miss. Midtraining was our last chance to close gaps in entities, camera angles, and styles.

Krea 2 illustration of a baobab viewed from below, its branches spreading across a pink sky.
Baobab · fisheye perspective from below
Krea 2 illustration of a red tiger pouncing on a boar amid bright jungle foliage.
Tiger and boar · subjects in motion
Krea 2 ink-style illustration of a samurai drawing a sword, seen from a low, wide-angle viewpoint.
Samurai · pose and wide-angle composition

Selected generations from Krea's official examples ↗.

Moodboards and Prompt Expansion

I built and trained the text prompt expander behind moodboards. This is text-to-image generation: the moodboard's images are interpreted in text space, rather than passed to the image model as a set of direct image references. The expander folds their visual direction into the user's prompt, so a board can combine disparate references. A selected style-reference image separately supplies texture and style.

I trained the prompt expander through SFT and RL, designing text rewards to complement the team's image rewards. Safety training addressed unintended IP leakage and unwanted nudity while preserving accurate anatomy and artistic expression.

Below, the same two subject prompts run through four moodboards.

Cyber Zine outputs · same two prompts · click to expand

Krea 2 horse portrait in blue-and-black halftone collage with sharp radiating graphic shapes.
“Portrait of a horse”
Krea 2 chameleon in a sweater, rendered with pixelated collage textures and retro computer graphics.
“Chameleon wearing a sweater”

Minimalist Ink outputs · same two prompts · click to expand

Krea 2 horse portrait drawn with sparse black ink lines on white.
“Portrait of a horse”
Krea 2 chameleon wearing a sweater, drawn small in sparse black ink lines with broad white negative space.
“Chameleon wearing a sweater”

Futurist Glam outputs · same two prompts · click to expand

Krea 2 horse portrait with luminous metallic makeup and a silver mane against black.
“Portrait of a horse”
Krea 2 chameleon wearing sunglasses and an orange knit sweater, perched on a reflective metal chair.
“Chameleon wearing a sweater”

Thermal Airbrush outputs · same two prompts · click to expand

Krea 2 horse portrait with glowing pink, orange, and blue edges against a dark background.
“Portrait of a horse”
Krea 2 chameleon in a pale ribbed sweater, rendered in glowing pink, orange, and blue gradients.
“Chameleon wearing a sweater”

Krea 2 technical report (coauthor) ↗ · Explore Krea 2 ↗

Other Research

ARC Challenge — Discrete Diffusion–Based Reasoning

2025

I hacked Qwen3-8B to solve ARC puzzles by filling in the easy parts first, instead of working in typewriter order. I built the training and evaluation pipeline to compare the two approaches.

Technical write-up · Hacker News discussion ↗

Private Federated Learning and On-Device Recommendations

Google Gboard

Phones encode words into compact sketches; secure aggregation combines them before noise is added to release frequent-word counts. I built private heavy-hitter tasks and custom TensorFlow Lite ops to run SecAggIBLT efficiently on Android, helping improve typing models, emoji search, and sticker suggestions.

Selected contributions
  • Shipped the team's first personalized on-device models: sticker recommendations using item-item collaborative filtering and user-item embeddings.
  • Built private heavy-hitter tasks with IBLTs, secure aggregation, and differential privacy. Used the data to improve typing models, emoji search, and sticker suggestions.
  • Built custom TensorFlow ops to run SecAggIBLT efficiently on Android phones through TensorFlow Lite.
  • Helped establish federated model evaluation and resolved barriers to private heavy-hitter tasks across teams.

AutoML for Advertising Models

Meta

I built AutoML systems for hundreds of advertising models, including feature-importance metrics that identified inputs that hurt performance. I led a cross-team effort to measure which AutoML changes helped, accounting for model size and training differences.

Personalization and Video Stylization

Co-founder and CTO · Later Comfy Org

At Drip.art, we adapted existing diffusion research into personalized portraits and video stylization before becoming Comfy Org. My work included automated LoRA/DreamBooth training pipelines, evaluation tools for personalized model quality, and infrastructure to serve hundreds of custom models concurrently.

Just for Fun

Kinect the Dots

Interactive art · 2015

A wall of about 1,900 LEDs that turns nearby people into live RGB silhouettes. I built the installation from the physical structure and wiring to the C++ depth-camera processing and microcontroller code.

Shown at Bay Area Maker Faire, Burning Man, and Santa Cruz Glow. Received two Editor’s Choice awards at Maker Faire.

Inside the installation

A Kinect tracks people; a MacBook sends the display data over USB to a Teensy microcontroller driving the LEDs. Two mirrored acrylic layers create an infinity-mirror effect.

Maker Faire project ↗

Blaze/Bazel Build Widget

Hardware · 2017

Cold builds could take 30–45 minutes, and I kept missing when they finished—or failed. I built a USB-connected LED widget to make the result hard to miss, then taught teammates to assemble and program their own.

How it works

A Python daemon polls the build server and sends status changes to a Teensy microcontroller. A ring of LEDs pulses red for failure and green for success, inside a laser-cut wood enclosure with a white acrylic diffuser.

Experience

Krea AI

September 2025 - Current

Joined as the fourth AI researcher on the small team behind Krea 2 and Krea 3. Worked across pretraining, midtraining, post-training, and prompt expansion, and on Krea 3's inference harness. Led data curation for both models.

Birchwood Software Corp.

Co-founder. Built a general-purpose RAG engine with custom retrieval and ranking, delivered through Slack. Sold it to its main customer.

Comfy Org (formerly Drip.Art)

January 2023 - 2025

Co-founder and CTO. Led a small engineering team and pre-seed fundraising, from personalized image generation to ComfyUI workflow infrastructure. Stayed on as an advisor after leaving the CTO role.

  • Built evaluations for personalized models and studied engagement across 10k+ users; brought ComfyUI's original author on as a co-founder and led the pivot to workflow infrastructure.
  • Built automated LoRA/DreamBooth training and served hundreds of custom models concurrently.
  • Built GPU orchestration across AWS, Lambda Labs, and CoreWeave. Delivered same-day fine-tuning and subsequent generation in under five minutes.

Meta

May 2022 - December 2022

Ads Core ML. Built and optimized AutoML systems; led cross-team work to measure the gains from different automated approaches.

Google

December 2016 - April 2022

Gboard — Federated Learning

July 2019 - April 2022

Worked on Gboard's modeling team and partnered across teams on private federated learning. Mentored teammates and gave tech talks on collaborative filtering.

Search — Frontend

December 2016 - July 2019

Improved mobile Search rendering through experiments; built libraries used by virtually every mobile query and async APIs. Supported shared components. Led Search Features Bootcamp sessions and managed two STEP interns whose work led to full-time offers.

Apple

May 2013 - December 2016

Built authoring and localization tools for the Tips app and interactive product tours in more than 20 languages. Led an exploratory support chatbot project, managed an intern, and mentored junior engineers.

Just For Fun

  • Technical blog: ML experiments and practical AI engineering, including how a simple document-reading agent outperformed our RAG pipeline. Read the blog.
  • Blaze/Bazel build widget (2017): Built an LED widget to flag finished or failed builds, then taught teammates to assemble and program their own.
  • Kinect the Dots (2015): Built an interactive installation with about 1,900 LEDs; received two Editor's Choice awards at the Bay Area Maker Faire ↗.

Education

Rochester Institute of Technology · 2009 - 2013
BS, Computer Science. Minors in Mathematics and Economics.