In this talk, I'll be exploring two related hypotheses. The first hypothesis is that our representation of the meaning of an utterance should make reference to the operation of an algorithm being executed by an agent interacting with the world. The agent can either be the speaker, or the person from whose perspective the content of the utterance is being described (for instance, the observer or protagonist in a narrative). I'll illustrate this idea by showing how the semantics of the connective `but' can be defined relative to a very high-level folk-psychological algorithm for interleaving action, perception and theorem-proving. I'll also discuss briefly how this algorithmic framework relates to recent frameworks for dynamic semantics. The second hypothesis is that lower level psychological accounts of the integration of action and perception can also be of use in modelling linguistic phenomena. I'll illustrate this by outlining a simple model of sensorimotor control, and sketching a possible application of this model in describing the semantics of progressive sentences. I'll conclude by considering whether these two examples have anything to do with each other; in other words, whether they make any common assumptions about the role of an algorithm modelling an agent's general cognitive and motor capabilities in the representation of linguistic phenomena.