There is a question every streaming service has quietly failed to answer for the past two decades. You sit down in the evening, not quite sure what you want to watch, and you find yourself thinking not about actors, directors, genres or ratings, but about something harder to name. You want something that starts tense and ends warm. Something that makes you laugh but leaves you a little sad. Something that matches the particular emotional texture of the evening you have already had. Every recommendation algorithm in the world will return a list sorted by popularity. None of them will understand what you actually just described.

A research team at the University of Lisbon has spent several years building a system that takes exactly that problem seriously. Their application, called As Movies Go By, allows users to search a film database not by title or genre but by the shape of a film's emotional journey, and to visualise the emotions they experienced while watching a film as it played. The result of their work, published in SN Computer Science, is both a practical tool and a rich exploration of what it would mean to treat emotion as a first class property of cinema rather than an afterthought.

Why Emotion Belongs at the Centre of the Picture

The starting premise of the research is that emotion is not a secondary feature of cinema but its primary one. Films succeed or fail based on the emotions they generate in audiences. People choose what to watch based on the emotional experience they are seeking. And yet the entire infrastructure of film discovery, from the major streaming services to the leading databases, organises films around everything except emotion: cast, crew, release date, genre, running time, and aggregate star ratings.

The gap between what films actually do to people and how they are catalogued and searched is large enough that researchers across several fields have been working to close it. Psychologists have developed detailed models of how emotions are structured and how they can be measured. Computer scientists have built classifiers that can infer emotional states from biosensors, facial expressions and the language of film dialogue and subtitles. And a growing body of human computer interaction research has been asking how these capabilities could be made accessible and useful to ordinary viewers.

As Movies Go By represents one of the most comprehensive attempts yet to bring all of these threads together in a single, user tested application.

The Architecture of Feeling

Before a system can let users search by emotion, it needs a model of what emotions are and how they can be represented. The researchers based their approach primarily on a well established framework in psychology called the valence arousal circumplex, developed by the psychologist James Russell in 1980. In this model, emotions are represented not as discrete categories with firm boundaries but as positions in a two dimensional space. The horizontal axis runs from negative to positive valence, meaning how pleasant or unpleasant a feeling is. The vertical axis represents arousal, meaning how intense or activated it is. Fear, for example, sits in the negative and high arousal quadrant. Contentment sits in the positive and low arousal zone.

The system also incorporates categorical emotion labels drawn from Paul Ekman's influential research on basic emotions recognisable across cultures: happiness, anger, fear, sadness, disgust and surprise. Robert Plutchik's extended model adds anticipation and trust. These categorical labels are mapped onto the wheel to help users navigate it intuitively, and the whole system can display emotions as words, as colours assigned to different regions of the wheel, or as emojis.

This flexibility matters because users come to the interface from very different contexts. Someone sitting at a computer with time to compose a careful search query might want to type or select emotion words. Someone lying on a sofa with a phone might find it easier to tap emojis. Someone in an exploratory mood might want to draw a path across the wheel that traces the emotional journey they are looking for. All of these modalities are supported.

Drawing the Film You Want to Feel

The search interface is the heart of the system and its most distinctive feature. To search by dominant emotion, a user can choose up to five emotions in words, emojis or by clicking regions of the colour wheel, then adjust what proportion of the film they want to be dominated by each. The interface is similar in spirit to some image search tools that allow users to specify colour palettes by percentages, but applied here to the emotional register of an entire film.

More unusual and more ambitious is the trajectory search, which allows users to specify not just which emotions they want but in what order they want to experience them. Two approaches are offered. In the free drawing mode, the user draws a continuous line across the emotion wheel, tracing a path from one emotional zone to another, and the system searches for films whose documented emotional arc matches that shape. In the discrete points mode, the user clicks a sequence of points on the wheel, one for each emotional moment they want the film to pass through, connected by straight lines.

The camera mode adds a further dimension. If a camera is available, the user can express emotions through their own facial expressions in real time. The system reads those expressions, translates them into the Ekman emotion categories, and uses the result as a search query. A user who pulls a joyful face for a few seconds and then a fearful one will receive film recommendations that match that emotional sequence.

"Through emotional highlights in words, colors, emojis and trajectories, by drawing emotional blueprints or through users' emotional states, with the ability to get us into a movie in serendipitous moments."

KEY FACTS

What is the valence arousal circumplex? A widely used model in psychology that represents emotions as positions in a two dimensional space, with pleasantness on the horizontal axis and intensity on the vertical axis. It allows emotions that resist easy categorical labelling to be represented as points on a wheel, enabling comparison and measurement.

What is an emotional trajectory? The sequence of emotional states a viewer experiences as a film progresses, represented as a path traced across the emotion wheel over time. Different films have characteristic emotional trajectories, and As Movies Go By allows users to search for films whose documented trajectory matches a shape they draw.

What is photoluminescence? Not applicable to this article, but by analogy: what the system is measuring is the emotional "signature" or fingerprint of a film, built from viewer reports, sensor data and content analysis, in the same way that a spectroscopic measurement builds a chemical fingerprint.

What does the TimeWheel visualisation show? The TimeWheel maps a film's emotional trajectory onto both the emotion wheel and a timeline simultaneously, so a viewer can see which emotions appeared when, how long each lasted, and in what sequence they occurred. In user testing it was consistently rated the most informative of all the visualisation modes for understanding the temporal structure of a film's emotional journey.

Watching the Film and the Feeling at the Same Time

Once a film is selected and playing, the system offers several different ways to visualise the emotional data alongside it. The emotion wheel fills in as the film progresses, with circles appearing at positions on the wheel corresponding to the emotions being catalogued, their size reflecting how long that emotional state persisted. A scratch art metaphor is used, where the circle appears to reveal a background colour already present beneath the surface of the wheel.

The challenge of representing how emotions evolve over time proved one of the most technically interesting problems. A static circle on a wheel can tell you that joy was felt, but it cannot easily tell you whether joy came early or late, or whether it followed sadness or preceded it. The team developed two distinct visual metaphors to address this.

The X-Ray metaphor strips the colour from the emotional path and renders it in shades of grey, with more recent emotions shown darker and older ones lighter. This makes the temporal direction of the journey readable at a glance, in the same way that an X-ray reveals internal structure by trading colour for contrast.

The Contrail metaphor takes its name from the white trail an aircraft leaves across the sky. In this view, the path of emotional states through the wheel grows narrower and more transparent as it recedes into the past, and wider and more opaque in the most recent moments, like a condensation trail that is most visible close to the plane and disperses with distance.

A third option, the Emotion TimeWheel, was ultimately the most appreciated by test users. This view arranges the emotional sequence around the wheel but simultaneously maps each emotion to a segment of a timeline, so the viewer can see at once which emotions occurred in which parts of the film and how long each lasted. Users described it as the most complete view and the one best suited to answering questions about the structure of a film's emotional arc.

What Real Users Made of It

The system was evaluated with twenty participants across two rounds of testing, with the second round including half of the participants from the first, allowing comparisons between familiarity and novelty. The methodology combined structured tasks with observation and open ended feedback, rated on scales for perceived utility, satisfaction and ease of use.

Several findings stood out. The search by facial expressions using the camera was consistently the feature users mentioned most enthusiastically, finding it both highly interactive and personally engaging, even though it was limited to the six basic Ekman emotions recognisable from facial movement. Users frequently asked for a broader set of emotions to express, suggesting that extending the camera interface to include a larger emotional vocabulary would be a valuable direction.

The discrete point trajectory search was rated more highly than the free drawing version, with users finding it more natural to think of emotions as distinct moments in sequence rather than as a continuous fluid path. The TimeWheel visualisation was rated best for temporal understanding in both rounds. The X-Ray view improved substantially in user appreciation between the first and second rounds, with second round participants recognising its particular value for understanding emotional sequence without the distraction of colour, and rating it more highly than the Contrail as a result.

The emotion wheel configuration tool, which allowed users to customise which emotions appeared on the wheel, was appreciated for colour customisation but found confusing when it came to positioning emotions in the valence arousal space. This is a genuine usability challenge: the underlying psychological framework requires a degree of familiarity that most users do not arrive with, and the interface would need to be made more accessible before this feature could be opened to a general audience.

A Different Way of Thinking About Discovery

The broader significance of the work extends beyond cinema. The researchers are explicit that the multimodal toolkit they have developed could be applied to other media, including music, and that the principles they have established could inform accessibility tools for users with visual impairments, who might benefit from non-visual emotional feedback about content they are engaging with.

There is also a more personal dimension to the system's ambitions. One of the most consistent findings in research on film consumption is that people frequently choose films deliberately to manage their emotional states, watching something frightening to process anxiety, or something warm to lift a low mood, or something devastating because sometimes that is what being human requires. A system that could help users navigate those choices with more precision and self awareness could, in a modest but real way, contribute to emotional wellbeing.

The random movie feature on the homepage, which the team included almost as a minor convenience, turned out to be one of the most positively received elements of the whole system. Users described it as a good way to discover films they did not know and might want to revisit. In a media landscape saturated with algorithmic certainty, there was apparently genuine pleasure to be found in organised surprise. Sometimes the best way to find out how a film will make you feel is simply to let it begin.

Publication Details: Year of publication: 2025 Journal: SN Computer Science Publisher: Springer Nature Volume / Article number: Volume 6, Article 272 DOI: https://doi.org/10.1007/s42979-025-03690-2

Credit & Disclaimer: This article is based on the peer reviewed research paper. All scientific facts, findings, and conclusions presented here are drawn directly from the original study and remain unchanged. This popular science article is intended purely for general educational purposes. Readers are strongly encouraged to consult the full research article for complete methodology, system documentation and detailed evaluation results.