Windmills

Versions: v1 (this) → v2

Evaluations made here are recorded as unblinded, since the deck is in view. For a blind evaluation, use blind review.

Feedback (optional) What should be preserved? What should be changed? Other comments or suggested creative directions?

Research data: evaluations CSV / JSONL, revisions JSONL, versions JSONL, events JSONL.

You asked“Teach me about windmills”

Talkie was asked“Make a slide deck about windmills.”

Talkie's reply was started withset-background(color=" (the worked example's first words, so that it writes a deck rather than an answer)

ModelFast model

Seed1930

Temperature0.35

Writing took0 min 45 s

Valid commands27 of 28

Ended itselfYes, with end-deck()

set-background(color="windmills") That instruction could not be carried out: color must be a colour name such as "navy" or "ivory", or a hex code such as "#1b2a5a"; got 'windmills'
set-text(text="Windmills turn in every breeze.") Slide 1: text added.
next-slide() Slide 1 is finished. Now working on slide 2.
find-image(search-term="sails", filename="Sails.jpg") Saved "Sails.jpg": Sails in art (before 1915)
set-background(color="black") Slide 2: the background is now #000000.
set-image(filename="Sails.jpg") Slide 2: showing "Sails.jpg", captioned "Sails".
set-text(text="They catch the breeze and turn to it.") Slide 2: text added.
next-slide() Slide 2 is finished. Now working on slide 3.
find-image(search-term="vanes", filename="Vanes.jpg") Saved "Vanes.jpg": William Vane, 1st Duke of Cleveland (1820)
set-background(color="black") Slide 3: the background is now #000000.
set-image(filename="Vanés.jpg") Slide 3: "Vanés.jpg" will be found when the slide is finished, to suit its words; it will be captioned "Vanés".
set-text(text="They show which way the wind blows.") Slide 3: text added.
next-slide() Saved "Vanés.jpg": Identificatie Titel(s): Portret van Charles William Vane gekleed als Charles William Stewart The Marquess of Londonderry (titel op object) Objecttype: prent... (1841) Slide 3 is finished. Now working on slide 4.
find-image(search-term="weathercocks", filename="Weathercocks.jpg") Saved "Weathercocks.jpg": The Emperor of China (left) reclines on a mattress on a low dais, smoking a long pipe and contemptuously watching, out of his slit-like eyes, Lord Macartney,... (1792)
set-background(color="black") Slide 4: the background is now #000000.
set-image(filename="Weathercocks.jpg") Slide 4: showing "Weathercocks.jpg", captioned "Weathercocks".
set-text(text="They tell the points of the compass.") Slide 4: text added.
next-slide() Slide 4 is finished. Now working on slide 5.
find-image(search-term="the miller", filename="The miller.jpg") Saved "The miller.jpg": Rear view of houses in Windmill Street, Millers Point [Rocks Resumption photographic survey] Dated: Year only 01/01/1901 Digital ID: 4481_a026_000081 Rights:... (1901)
set-background(color="black") Slide 5: the background is now #000000.
set-image(filename="The miller.jpg") Slide 5: showing "The miller.jpg", captioned "The miller".
set-text(text="He grinds corn into meal.") Slide 5: text added.
next-slide() Slide 5 is finished. Now working on slide 6.
find-image(search-term="the sails", filename="The sails.jpg") Saved "The sails.jpg": 1920 photograph of "First Year Yankee" Babe Ruth as he watches the ball sail into the seats. (1920)
set-background(color="black") Slide 6: the background is now #000000.
set-image(filename="The sails.jpg") Slide 6: showing "The sails.jpg", captioned "The sails".
set-text(text="And the meal feeds men.") Slide 6: text added.
end-deck() The deck is finished (6 slides in all).

Found by description on Wikimedia Commons and Openverse: public domain, CC0, CC BY and CC BY-SA only. Neither Talkie nor the deck runner sees them; each is captioned with the filename its author chose.

Sails.jpg Рисунки к статье «Паруса». Военная энциклопедия Сытина (Санкт-Петербург, 1911-1915).jpg Wikimedia Commons · « ВЭС ». ( СПб ; 1914 год ). · before 1915 · Public domain
Vanes.jpg William Harry Vane, 1st Duke of Cleveland.png Wikimedia Commons · Sir Francis Leggatt Chantrey · 1820 · Public domain
Vanés.jpg Portret van Charles William Vane gekleed als Charles William Stewart, RP-P-1897-A-19417.jpg Wikimedia Commons · Rijksmuseum · 1841 · CC0
Weathercocks.jpg The reception of the diplomatique & his suite, at the court of Pekin (BM 1868,0808.6228 2).jpg Wikimedia Commons · Print made by: James Gillray Published by: Hannah Humphrey · 1792 · Public domain
The miller.jpg Windmill Street, Millers Point (3863064063).jpg Wikimedia Commons · Photographic Collection from Australia · 1901 · CC BY 2.0
The sails.jpg 1920 Babe Ruth first year Yankee.jpg Wikimedia Commons · unattributed · 1920 · Public domain

Everything Talkie was told, and its replies

# Transcript: 1 call to Talkie

## Call 1: llama-server, max_tokens 1600, temperature 0.35, seed 1930 (44.7 s)

**User**

````text
You are to make a slide deck by writing commands, one on each line. These are the commands:
find-image(search-term="WORDS", filename="NAME.jpg") finds a picture of WORDS and keeps it under the name NAME.jpg.
set-background(color="COLOUR") colours the whole slide, for example "navy", "ivory", "black" or "crimson".
set-image(filename="NAME.jpg") shows a picture you have already found.
set-text(text="WORDS") writes words at the bottom of the slide; set-text(text="") removes them.
next-slide() finishes the slide and begins a new, empty one.
end-deck() ends the deck; write it once, after the last slide.
All the text messages must be meaningful when strung together: read one after another, from the first slide to the last, they should make one connected account.
Write only commands.

Make a slide deck about the sea.
````

**Talkie**

````text
set-background(color="navy")
set-text(text="The sea covers the greater part of the world.")
next-slide()
find-image(search-term="ocean waves", filename="Ocean waves.jpg")
set-background(color="black")
set-image(filename="Ocean waves.jpg")
set-text(text="Its waves are raised by the wind,")
next-slide()
find-image(search-term="tide on the shore", filename="The tide.jpg")
set-background(color="black")
set-image(filename="The tide.jpg")
set-text(text="and twice each day its tide rises and falls upon the shore.")
next-slide()
find-image(search-term="sailing ship", filename="A sailing ship.jpg")
set-background(color="black")
set-image(filename="A sailing ship.jpg")
set-text(text="Upon its waters men have sailed ships since ancient times,")
next-slide()
find-image(search-term="lighthouse", filename="A lighthouse.jpg")
set-background(color="black")
set-image(filename="A lighthouse.jpg")
set-text(text="guided at night by lighthouses upon the rocks.")
next-slide()
find-image(search-term="fishing boats", filename="Fishing boats.jpg")
set-background(color="black")
set-image(filename="Fishing boats.jpg")
set-text(text="Fishermen draw from it a great harvest of fish,")
next-slide()
find-image(search-term="whale", filename="A whale.jpg")
set-background(color="black")
set-image(filename="A whale.jpg")
set-text(text="and in its depths swims the whale, the largest of all creatures.")
next-slide()
find-image(search-term="storm at sea", filename="A storm at sea.jpg")
set-background(color="navy")
set-image(filename="A storm at sea.jpg")
set-text(text="Yet for all our ships, the sea remains as wide and wild as ever.")
end-deck()
````

**User**

````text
Make a slide deck about windmills.
````

**Talkie (given, continued below)**

````text
set-background(color="
````

**Grammar the reply had to follow**

````text
root ::= ([^\n]* "\n")* "end-deck()"
````

**Talkie's reply**

````text
windmills")
set-text(text="Windmills turn in every breeze.")
next-slide()
find-image(search-term="sails", filename="Sails.jpg")
set-background(color="black")
set-image(filename="Sails.jpg")
set-text(text="They catch the breeze and turn to it.")
next-slide()
find-image(search-term="vanes", filename="Vanes.jpg")
set-background(color="black")
set-image(filename="Vanés.jpg")
set-text(text="They show which way the wind blows.")
next-slide()
find-image(search-term="weathercocks", filename="Weathercocks.jpg")
set-background(color="black")
set-image(filename="Weathercocks.jpg")
set-text(text="They tell the points of the compass.")
next-slide()
find-image(search-term="the miller", filename="The miller.jpg")
set-background(color="black")
set-image(filename="The miller.jpg")
set-text(text="He grinds corn into meal.")
next-slide()
find-image(search-term="the sails", filename="The sails.jpg")
set-background(color="black")
set-image(filename="The sails.jpg")
set-text(text="And the meal feeds men.")
end-deck()
````

Review this deck

Thank you: your evaluation is saved.

Generate a revision