Events

Control Mechanisms for Generative Models

The compelling outputs of modern generative models have motivated widespread efforts to deploy these models as tools in commercial products and user-facing applications. For a model to be an effective tool, we must be able to control its outputs. Users of a model require control in order to instruct the model to generate outputs that meet their needs. A model provider also requires control, in order to oversee and regulate the behavior of the model. In this talk, John Thickstun describes his work on methods that improve the controllability of generative models. First, Thickstun describes the Anticipatory Music Transformer: a generative model of music with a novel control mechanism called anticipation, which allows a user to apply fine-grained control while generating long sequences of music. Second, Thickstun shows how to control a pre-trained model of images or audio to generate outputs subject to constraints, and apply this method to source separation: “the cocktail party problem.” Third, Thickstun demonstrates a mechanism for watermarking the outputs of a language model, without altering the distribution of text generated by the model. Thickstun concludes with a vision for future work on controllable generation, and ways that more nuanced control mechanisms can facilitate the use and governance of generative models.