Dave Howcroft<p>It's been a while since I wrote an <a href="https://mastodon.social/tags/introduction" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>introduction</span></a> for folks who are <a href="https://mastodon.social/tags/NewHere" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NewHere</span></a>.</p><p>I'm a <a href="https://mastodon.social/tags/queer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>queer</span></a> <a href="https://mastodon.social/tags/Catholic" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Catholic</span></a> computational <a href="https://mastodon.social/tags/linguist" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>linguist</span></a> who works on <a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a> and lives in <a href="https://mastodon.social/tags/Scotland" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Scotland</span></a>. An <a href="https://mastodon.social/tags/immigrant" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>immigrant</span></a> multiple times over, I was born in <a href="https://mastodon.social/tags/SouthAfrica" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SouthAfrica</span></a>, grew up in the USA (in <a href="https://mastodon.social/tags/Florida" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Florida</span></a>), did my PhD in <a href="https://mastodon.social/tags/Germany" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Germany</span></a>, and moved here in 2019. I enjoy <a href="https://mastodon.social/tags/VideoGames" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>VideoGames</span></a>, <a href="https://mastodon.social/tags/VideoEssays" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>VideoEssays</span></a>, <a href="https://mastodon.social/tags/OpenMic" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenMic</span></a> nights, and <a href="https://mastodon.social/tags/running" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>running</span></a>. I happen to be <a href="https://mastodon.social/tags/nonbinary" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>nonbinary</span></a> and love <a href="https://mastodon.social/tags/languages" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>languages</span></a>.</p>
Recent searches
No recent searches
Search options
Only available when logged in.
heads.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
We're like licorice. Not everybody likes licorice, but the people who like licorice really like licorice.
Administered by:
Server stats:
151active users
heads.social: About · Profiles directory · Privacy policy
Mastodon: About · Get the app · Keyboard shortcuts · View source code · v4.3.6
#NaturalLanguageGeneration
0 posts · 0 participants · 0 posts today
Dave 🦔<p>A wee <a href="https://tech.lgbt/tags/introduction" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>introduction</span></a> since there are lots of folks joining or rejoining the fediverse in general and Mastodon in particular. </p><p>I'm Dave, a <a href="https://tech.lgbt/tags/queer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>queer</span></a> <a href="https://tech.lgbt/tags/Catholic" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Catholic</span></a> who runs and sings for fun. I play <a href="https://tech.lgbt/tags/VideoGames" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>VideoGames</span></a> (currently <a href="https://tech.lgbt/tags/Titanfall2" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Titanfall2</span></a>) and watch a lot of <a href="https://tech.lgbt/tags/Dropout" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Dropout</span></a> (<a href="https://tech.lgbt/tags/Dimension20" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Dimension20</span></a>, <a href="https://tech.lgbt/tags/PlayItByEar" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PlayItByEar</span></a>, <a href="https://tech.lgbt/tags/GameChanger" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>GameChanger</span></a> are faves), <a href="https://tech.lgbt/tags/VideoEssays" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>VideoEssays</span></a>, and <a href="https://tech.lgbt/tags/anime" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>anime</span></a>.</p><p>I recently went to my first <a href="https://tech.lgbt/tags/burn" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>burn</span></a> (not <a href="https://tech.lgbt/tags/BurningMan" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>BurningMan</span></a>, tho) and am digging the inclusive accepting culture. </p><p>At work I'm a <a href="https://tech.lgbt/tags/researcher" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>researcher</span></a> in <a href="https://tech.lgbt/tags/ComputationalLinguistics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ComputationalLinguistics</span></a> focused on <a href="https://tech.lgbt/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a> and <a href="https://tech.lgbt/tags/psycholinguistics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>psycholinguistics</span></a> methods. You could say my dayjob is trying to develop a more ethical <a href="https://tech.lgbt/tags/ChatGPT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ChatGPT</span></a></p><p>Feel free to ask me questions about <a href="https://tech.lgbt/tags/Mastodon" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Mastodon</span></a> if you are <a href="https://tech.lgbt/tags/NewHere" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NewHere</span></a></p>
Dave Howcroft<p>This is not to say, however, that I think these models are useless. I think the interesting question is how to integrate these models into systems that express a particular meaning, a la data-to-text <a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a>. Whether this involves <a href="https://mastodon.social/tags/PromptEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>PromptEngineering</span></a>, integrating them into the decoder for <a href="https://mastodon.social/tags/seq2seq" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>seq2seq</span></a> models, or some other more clever application remains to be seen. I am looking forward to seeing how <a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>LLM</span></a>/s get used for <a href="https://mastodon.social/tags/NLG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLG</span></a> going forward.</p>
Dave Howcroft<p>In the years since then there has been an explosive growth in interest in <a href="https://mastodon.social/tags/TextGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>TextGeneration</span></a>, but usually not grounded in expressing a particular meaning, which was historically a priority for work in <a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a>. This excitement came from the relative fluency of <a href="https://mastodon.social/tags/LargeLanguageModels" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>LargeLanguageModels</span></a> (<a href="https://mastodon.social/tags/LLM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>LLM</span></a>) and <a href="https://mastodon.social/tags/Transformer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Transformer</span></a> models in particular, which did an impressive job of continuing an initial utterance.</p>
Dave Howcroft<p>Around 2015 and 2016 we saw sequence-to-sequence (<a href="https://mastodon.social/tags/seq2seq" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>seq2seq</span></a>) models applied to data-to-text <a href="https://mastodon.social/tags/NLG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLG</span></a> for the first time. These models were trained end-to-end and were very exciting because it raised the prospect of reducing the amount of hand-crafted <a href="https://mastodon.social/tags/GrammarEngineering" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>GrammarEngineering</span></a> one would have to do to create a <a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a> system.</p>
Dave Howcroft<p>Since the <a href="https://mastodon.social/tags/NeuralNetwork" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NeuralNetwork</span></a> revolution in <a href="https://mastodon.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MachineLearning</span></a> came to <a href="https://mastodon.social/tags/NLP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLP</span></a> (<a href="https://mastodon.social/tags/NaturalLanguageProcessing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageProcessing</span></a>/#NLProc), these tools have also become very common in <a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a>. Of course, in the late 90s folks were already incorporating statistical information into their <a href="https://mastodon.social/tags/NLG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLG</span></a> systems in an approach called 'overgenerate-and-rank', where rules were underspecified and produced grammatical and ungrammatical utterances and one relied on n-gram frequencies to rank the possible outputs correctly</p>
Dave Howcroft<p><a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a>/#NLG has often relied on rule-based approaches to generation with a pipeline of processes. For example, a content selection module might process raw data to decide what is worth mentioning, with the result being fed into a sentence planning module to determine how that information should be expressed broadly (for example, which words to use, how to group the info into sentences, etc), and finally that sentence plan going through 'surface realisation' to become text</p>
Dave Howcroft<p>But <a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a> can broadly apply to all sorts of tasks. Even <a href="https://mastodon.social/tags/MachineTranslation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MachineTranslation</span></a> (<a href="https://mastodon.social/tags/MT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>MT</span></a>) is a kind of <a href="https://mastodon.social/tags/NLG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLG</span></a>. And other text-to-text transformations like summarisation and style-transfer also fall under the umbrella of what I call big-tent <a href="https://mastodon.social/tags/NLG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLG</span></a>.</p>
Dave Howcroft<p>For my work, I usually use <a href="https://mastodon.social/tags/NLG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLG</span></a> to refer to so-called data-to-text generation, where we are transforming some *non-linguistic* representation of information into a natural language utterance (whether that's spoken or written, long or short, formal or informal, etc)</p><p><a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a> <a href="https://mastodon.social/tags/data2text" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>data2text</span></a> <a href="https://mastodon.social/tags/d2t" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>d2t</span></a></p>
Dave Howcroft<p>Every day that I remember to do it, I'm going to explain what I mean by one of the hashtags in my profile. <a href="https://mastodon.social/tags/WhatHashtagsMeanToMe" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>WhatHashtagsMeanToMe</span></a></p><p>I'm at the office, so today is about professional stuff: <a href="https://mastodon.social/tags/NaturalLanguageGeneration" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NaturalLanguageGeneration</span></a> (<a href="https://mastodon.social/tags/NLG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NLG</span></a>) involves developing computer systems which can express information in natural language (that is, human languages). There are a bunch of different approaches to NLG and a lot of different tasks that can be considered a part of big-tent NLG.</p>
ExploreLive feeds
Mastodon is the best way to keep up with what's happening.
Follow anyone across the fediverse and see it all in chronological order. No algorithms, ads, or clickbait in sight.
Create accountLoginDrag & drop to upload