The integration of textual and visual information is fundamental to the way people communicate. Communication succeeds even though visual and spatial representations are not necessarily wired to syntax and conventions, and do not always replicate appearance. Our work explores the potential of natural language techniques for modeling multimodal communication. We present two case studies; diagrams and visual explanations. Both diagrams and visual explanations are at the margin of and connected to traditional resources of natural language theorizing. Similar to language, diagrams tend to abstract and schematize. Visual explanations also include instructional images that closely coordinate with the paired utterances to get their message across. We assess the effectiveness of our approach through crowdsourcing and machine learning methodology. Our results suggest that natural language techniques provide a useful foundation for reengaging with visual communicative artifacts.