I too have found GPT-4 very helpful and often proficient in surprising ways, but then been utterly disappointed at how rote it is with any creative task. My experience seems to be backed up by the fact that it aced virtually every standardized test, except AP English where it performed abysmally. https://twitter.com/rickyflows/status/1635693600272162817/photo/1
Nat Eliason had an interesting article on good writing having a hallucinatory aspect--where the reader gets into a flow state that seems almost as if they are writing the words as they read them. https://blog.nateliason.com/p/great-writing-invisible. GPT-4 seems awful at this. It is just so obvious in every moment that this is an LLM doing a very good job of rapidly responding to queries.
Also, please don't take GPT-4's second criticism of your writing. Honestly if you had "clear topic sentences, smooth transitions, and concise paragraphs" I very much doubt I would be interested in your writing. I, and I suspect many others, enjoy the fact that there is an ineffable nature to how you write. I don't expect or want you to efficiently take me through a very polished argument; I want to be entertained, sometimes confused, and ultimately inspired that another human I have never met has found a way to dump some of their mindjuice into my head.
i do wonder on what do different-tasted people (or tasteless people) think, if they'd be able to, A/B tested, notice the difference between sasha and GPT. I think I could, but I'm also not 100% confident, the bot could probably fool me sometimes
I have to agree with Spencer, above, though I suppose interesting writing and writing that defends an argument clearly and formally can at times overlap (at least, that’s what I think the suggestion about “clear topic sentences” might be referring to).
I think this discussion comes back to what Edward Tian, the Princeton student who developed an app to help recognize if a text was written by AI or by a human, refers to as “perplexity” and “burstiness.” We humans tend to be more random, and rhythmically varied, in our thought processes (perception? experience? all of the above?) and it shows in the language we use, evidently. A kind of indeterminate aesthetic quality of aliveness.
Personally though, I think I’d have a hard time “collaborating” with an AI on creative writing. If it’s a tool, then at best it’s a spellcheck and passive-aggressive email coach (mine already tells me to use the active voice more and not qualify my statements, which I resent). If it’s as good as a human writer, on the other hand, I fear I’d keep asking myself, who is writing? And there wouldn’t be a good way to tell--is it just very good at imitating me, or what. Then again, maybe that kind of discomfort can be interesting, and even productive.
Hi Sasha. I’m curious whether you explicitly take inspiration from Nietzsche in terms of style? Something about these passages reminded of the ending of *Beyond Good and Evil*, which is one of my favorite bits of text:
“Alas! what are you, after all, my written and painted thoughts! Not long ago you were so variegated, young and malicious, so full of thorns and secret spices, that you made me sneeze and laugh and now? You have already doffed your novelty, and some of you, I fear, are ready to become truths, so immortal do they look, so pathetically honest, so tedious! And was it ever otherwise? What then do we write and paint, we mandarins with Chinese brush, we immortalisers of things which LEND themselves to writing, what are we alone capable of painting? Alas, only that which is just about to fade and begins to lose its odour! Alas, only exhausted and departing storms and belated yellow sentiments! Alas, only birds strayed and fatigued by flight, which now let themselves be captured with the hand with OUR hand! We immortalize what cannot live and fly much longer, things only which are exhausted and mellow! And it is only for your AFTERNOON, you, my written and painted thoughts, for which alone I have colours, many colours, perhaps, many variegated softenings, and fifty yellows and browns and greens and reds; but nobody will divine thereby how ye looked in your morning, you sudden sparks and marvels of my solitude, you, my old, beloved- EVIL thoughts!”
Mar 17, 2023·edited Mar 17, 2023Liked by Sasha Chapin
David Samuels describing a similar stylistic tendency: "I think if you look both on the line and structural levels, one of the effects I’m trying to model in my writing is a habit of making connections between kinds of discourse that are usually separate. And so if you break down good sentences of mine you’ll find some high falutin’ turn of phrase and some slang-y turn of phrase. The energy comes from the way that they are jammed together, this sort of elevator effect where you’re in the basement and then you go to the eighty-fifth floor and you do that in seven words, which is generally how I amuse myself." https://thenewjournalatyale.com/2012/09/a-conversation-with-david-samuels/
Interesting that this is still hard for an AI, but then again also interesting that it seems to be a late-developing skill for a lot of writers.
To me it seems like there is a more distinct experiential difference between reading your writing and GPT's in these examples—yours feel like being efficiently shown around a world (or your version of what the world is like), whereas GPT's feel like there isn't a world so much as a slightly dizzying hash of world-like bits. Not sure if this is actually going on.
I too have found GPT-4 very helpful and often proficient in surprising ways, but then been utterly disappointed at how rote it is with any creative task. My experience seems to be backed up by the fact that it aced virtually every standardized test, except AP English where it performed abysmally. https://twitter.com/rickyflows/status/1635693600272162817/photo/1
Nat Eliason had an interesting article on good writing having a hallucinatory aspect--where the reader gets into a flow state that seems almost as if they are writing the words as they read them. https://blog.nateliason.com/p/great-writing-invisible. GPT-4 seems awful at this. It is just so obvious in every moment that this is an LLM doing a very good job of rapidly responding to queries.
Also, please don't take GPT-4's second criticism of your writing. Honestly if you had "clear topic sentences, smooth transitions, and concise paragraphs" I very much doubt I would be interested in your writing. I, and I suspect many others, enjoy the fact that there is an ineffable nature to how you write. I don't expect or want you to efficiently take me through a very polished argument; I want to be entertained, sometimes confused, and ultimately inspired that another human I have never met has found a way to dump some of their mindjuice into my head.
haha i miss u on twitter btw
good post, i like the approach of taste
i do wonder on what do different-tasted people (or tasteless people) think, if they'd be able to, A/B tested, notice the difference between sasha and GPT. I think I could, but I'm also not 100% confident, the bot could probably fool me sometimes
I have to agree with Spencer, above, though I suppose interesting writing and writing that defends an argument clearly and formally can at times overlap (at least, that’s what I think the suggestion about “clear topic sentences” might be referring to).
I think this discussion comes back to what Edward Tian, the Princeton student who developed an app to help recognize if a text was written by AI or by a human, refers to as “perplexity” and “burstiness.” We humans tend to be more random, and rhythmically varied, in our thought processes (perception? experience? all of the above?) and it shows in the language we use, evidently. A kind of indeterminate aesthetic quality of aliveness.
Personally though, I think I’d have a hard time “collaborating” with an AI on creative writing. If it’s a tool, then at best it’s a spellcheck and passive-aggressive email coach (mine already tells me to use the active voice more and not qualify my statements, which I resent). If it’s as good as a human writer, on the other hand, I fear I’d keep asking myself, who is writing? And there wouldn’t be a good way to tell--is it just very good at imitating me, or what. Then again, maybe that kind of discomfort can be interesting, and even productive.
This tweet feels related to this post:
https://twitter.com/jachaseyoung/status/1636650321199136769
Hi Sasha. I’m curious whether you explicitly take inspiration from Nietzsche in terms of style? Something about these passages reminded of the ending of *Beyond Good and Evil*, which is one of my favorite bits of text:
“Alas! what are you, after all, my written and painted thoughts! Not long ago you were so variegated, young and malicious, so full of thorns and secret spices, that you made me sneeze and laugh and now? You have already doffed your novelty, and some of you, I fear, are ready to become truths, so immortal do they look, so pathetically honest, so tedious! And was it ever otherwise? What then do we write and paint, we mandarins with Chinese brush, we immortalisers of things which LEND themselves to writing, what are we alone capable of painting? Alas, only that which is just about to fade and begins to lose its odour! Alas, only exhausted and departing storms and belated yellow sentiments! Alas, only birds strayed and fatigued by flight, which now let themselves be captured with the hand with OUR hand! We immortalize what cannot live and fly much longer, things only which are exhausted and mellow! And it is only for your AFTERNOON, you, my written and painted thoughts, for which alone I have colours, many colours, perhaps, many variegated softenings, and fifty yellows and browns and greens and reds; but nobody will divine thereby how ye looked in your morning, you sudden sparks and marvels of my solitude, you, my old, beloved- EVIL thoughts!”
David Samuels describing a similar stylistic tendency: "I think if you look both on the line and structural levels, one of the effects I’m trying to model in my writing is a habit of making connections between kinds of discourse that are usually separate. And so if you break down good sentences of mine you’ll find some high falutin’ turn of phrase and some slang-y turn of phrase. The energy comes from the way that they are jammed together, this sort of elevator effect where you’re in the basement and then you go to the eighty-fifth floor and you do that in seven words, which is generally how I amuse myself." https://thenewjournalatyale.com/2012/09/a-conversation-with-david-samuels/
Interesting that this is still hard for an AI, but then again also interesting that it seems to be a late-developing skill for a lot of writers.
Cool. The technology writes like a talented teenager. I'm not sure I want it to grow up. But something will inevitably evolve.
To me it seems like there is a more distinct experiential difference between reading your writing and GPT's in these examples—yours feel like being efficiently shown around a world (or your version of what the world is like), whereas GPT's feel like there isn't a world so much as a slightly dizzying hash of world-like bits. Not sure if this is actually going on.