Arter
- TIMELINE
- 2020
- TEAM
- Rodney Edwards · Siladityaa Sharma
- PARTNERS
- ArtCenter College of Design — Graduate ML Immersion Lab Instructors: Jenny Rodenhouse & Todd Masilko · Advisors: Krystina Castella, Robbie Nock
Arter is an AI-powered storyboarding tool. Its agent, Art, listens for a user's input and, through natural language processing, parses each word into an image object — turning a spoken or written idea into a visual storyboard in real time. Built over two terms in ArtCenter's graduate machine-learning immersion lab, the project paired a complete product concept with a working ML pipeline that achieved genuine text-to-sketch generation.
Why
As humans, the need to communicate is paramount. Our primary tool is language — most often written or spoken. But more often than not, much is lost in translation. That's why they say a picture is worth a thousand words. Arter started from a simple frustration: the distance between the idea in your head and your ability to show it to someone else. Sketching is a skill not everyone has, and even people who can sketch don't always feel comfortable putting rough work in front of a client.
The brief
How might we leverage the power of machine learning and natural language processing to enhance the way we communicate and visualize our ideas? The project came out of a graduate machine-learning immersion lab at ArtCenter College of Design. The goal of the class was to gain a deeper understanding of ML by exploring its associated technologies — and to let that discovery drive the concept, rather than starting from a fixed idea. Over two terms it spanned the core product experience, a working prototype, and research, alongside business viability, market segmentation, and a go-to-market plan.
Meet the users
We designed for two kinds of visual thinker. Sam is a Creative Director in customer experience — "the visual thinker." His agency is working with Blue Bottle Coffee on revamping their cafe space, and he needs to visualize the user journey to find opportunities. Normally he'd lean on the design team to imagine his storyboard, but they're busy, and he isn't comfortable presenting his own sketches to the client. Mary is a Product Manager — "the product launch." Her company is announcing a new product roadmap and leadership wants a sizzle reel to attract a new demographic. Mary needs to hand her media team a storyboard that communicates the marketing message and highlights new ways to use the product.
The concept
Art is the agent at the center of Arter. It listens for the user's input and, through natural language processing, parses each word into an image object using image-association analysis. You type or speak a scene — "a person sitting in a cafe, working on their laptop, drinking coffee" — and Art breaks the sentence into its visual nouns and actions, retrieves a matching illustration for each, and lays them onto the canvas as an editable storyboard frame. The agent does the drawing so the thinker can stay in the flow of the idea.
Type a scene
The core interaction is a single input bar. As you type, Art highlights the words it recognizes as objects and actions — "person," "cafe," "laptop," "coffee" — and offers them as draggable elements. Drag a word onto the frame and the matching illustration drops into place on a perspective ground plane you can arrange freely. Art isn't a black box that hands back a finished image; it's a collaborator. A conversational side panel lets you ask it to adjust, swap, or explain what it placed and why.
The technology
The concept was backed by a real, working ML pipeline — not a mockup. For the visuals, we used a SPADE-COCO model chained with an Edge-Detection model that converts simple doodles into sketches, and used AttGAN to achieve genuine text-to-sketch generation. To prototype the conversational experience, we built a model using AWS Lex and a sandboxed experience inside Runway ML, driven by Apple's dictation. Standing up the actual models — YOLOv3, Keras-OCR, SketchToPix, face trackers — is what grounded the product decisions in what the technology could really do.
Craft & system
Arter is set in Archia, a typeface chosen for the idea at the project's core — "where machine meets human." The interface pairs that with a calm, near-monochrome canvas and one bright accent: the rainbow halo around Art, the only place color enters the system. Much of the design work happened in the lock guides — the redlined specs for every screen, micro-interaction, and object state — so the system would hold together as it grew beyond the few screens we could show.
Process
The project was grounded in hands-on research — journey mapping, storyboarding workshops, and rounds of whiteboard sketching to understand how visual thinkers actually move from idea to artifact. We mapped where the friction lives — the moment of translation between language and image — and designed the agent to live exactly there. The output wasn't just an interface; it was a point of view on agentive technology: software that doesn't wait for commands but actively collaborates, anticipating what the visual thinker needs next.
Key takeaways
Four insights came out of the work: Machine learning can off-load the pain point of initial design brainstorming. It can also drive accessibility-first design and help eliminate edge cases. Designers make better decisions when they understand the technology powering their projects. And — most importantly — machine learning shouldn't replace the designer; it should elevate their decisions and assist them. If we'd had more time, we'd have kept refining the UX flow, covered more edge cases, built out features like the object context menu and brush tool, and shipped a fully interactive prototype.
Recognition
Arter was recognized internationally: Gold at the MUSE Design Awards 2020; Silver in Interface Design and Bronze in Web Design at the 2020 International Design Awards; featured at Dutch Design Week and on Bestfolio; and nominated for the A' Design Awards, Fast Company, Core77, and the Spark Awards. A project co-created with Rodney Edwards at ArtCenter College of Design, in the graduate ML immersion lab led by Jenny Rodenhouse and Todd Masilko, with advisors Krystina Castella and Robbie Nock.
