どすえのブログ

ソフトウェア開発ブログ

Reflexion：動的な記憶と自己反省を持つ自律的なLLMエージェント

論文 arxiv.org

実装 github.com

はじめに

自己反省をすることで、人間は試行錯誤を繰り返しながら、新しい問題を効率的に解決することができます。この研究ではLLMエージェントに動的な記憶(scratchpad)と自己反省能力(reflections)を付与し、既存の推論トレースとタスク固有の行動選択能力を強化するアプローチであるReflexionを提案しています。

また、完全自動化を実現するための、簡単かつ効果的なヒューリスティックを導入しています。このアプローチを評価するために、AlfWorld環境での意思決定タスクとHotPotQA環境での知識集約的な検索ベースの質問応答タスクにおけるエージェントの能力を評価しところ、それぞれ97%と51%の成功率を記録し従来の研究を圧倒しています。

概要

ReflexionはReActというフレームワークに自己反省機構(Reflect)を組み合わせた構造を取ります。ReActはLLMに推論と行動を繰り返し行わせ、タスクを実行させるためのフレームワークです。(詳しくは以下の参考文献を参照)

Reflexionは下図のようになっています。ReActでは、環境(Environment)に対して行動(Action)を行うと結果を観測(Observation)できます。また結果の良し悪しに応じて報酬(Reward)を設定することができます。

これらの行動、観測、報酬の履歴をヒューリスティック(Heuristic)に入力します。ヒューリスティックは平たく言えば、入力された履歴が反省するべき対象か判定するゲートのような役割を果たします。反省すべき場合というのは、得られた最終結果が所望のものと違う場合や、結果を得るまでに試行回数が多すぎる場合などです。

情報検索型QAシステムでの実例

データセット

ここではHotPotQAを解く例を扱います。 HotPotQAは、Wikipediaベースの質問と答えのペアを持つデータセットであり、質問応答（QA）システムの研究や評価に使用されています。このデータセットは、正確な回答を得るために複数のサポート文書を統合することが必要な質問を含んでいます。

エージェントの回答方法

エージェントはThought, Action, Observationの3段階のステップ織り交ぜながら、質問回答タスクを解くことを求められます。Thoughtは現在の状況を踏まえて次に取るべき行動を推論します。この推論に基づいてActionで行動を取ります。Observationでは環境に対するActionの結果を取得します。

またActionには具体的に3種類の行動が存在します。

  1. Search[entity]：Wikipediaで正確なentityを検索し、存在すれば最初のパラグラフを返す。存在しない場合は、検索するためにいくつかの類似のentityを返す
  2. Lookup[keyword]：Searchで見つかった最後の文章に含まれるkeywordを含む次の文章を返す
  3. Finish[answer]：answerを返し、タスクを終了させる

react_reflect_agent_prompt

Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types: 
(1) Search[entity], which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search.
(2) Lookup[keyword], which returns the next sentence containing keyword in the last passage successfully found by Search.
(3) Finish[answer], which returns the answer and finishes the task.
You may take as many steps as necessary.
Here are some examples:
{examples}
(END OF EXAMPLES)

{reflections}

Question: {question}{scratchpad}

このプロンプトには以下の変数が埋め込まれています。

  • examples：Think→Act→Observeの一連の流れの例示。内容は本記事下部に掲載してあります
  • reflections：前回の試行から得られた反省結果。反省結果は以下のreflect_promptを使って得られる
  • question：現在与えられている問い
  • scratchpad：現在までのThink→Act→Observe過程の記録

ThinkとActは共にprompt_agent()というメソッドが担当します。

# Think
self.scratchpad += f'\nThought {self.step_n}:'
self.scratchpad += ' ' + self.prompt_agent()

# Act
self.scratchpad += f'\nAction {self.step_n}:'
action = self.prompt_agent()
self.scratchpad += ' ' + action
action_type, argument = parse_action(action)

prompt_agent()の中身は以下のようになっています。_build_agent_prompt()にてexamples reflections question scratchpadを与えたプロンプトを生成し、LLMを駆動します。ここでreflectionsに有用な情報が含まれていれば質の高いThinkが可能になるということですね。

def prompt_agent(self) -> str:
    return format_step(self.llm(self._build_agent_prompt()))

...

def _build_agent_prompt(self) -> str:
    return self.agent_prompt.format(
                        examples = self.react_examples,
                        reflections = format_reflections(self.reflections),
                        question = self.question,
                        scratchpad = self.scratchpad)

reflectionsを生成するプロンプトは以下の通りです。

  • examples：反省方法の例示。内容は本記事下部に掲載してあります
  • question：前回与えられていた問い
  • scratchpad：前回の試行におけるThink→Act→Observe過程の記録

reflect_prompt

You are an advanced reasoning agent that can improve based on self refection. You will be given a previous reasoning trial in which you were given access to an Docstore API environment and a question to answer. You were unsuccessful in answering the question either because you guessed the wrong answer with Finish[<answer>], or you used up your set number of reasoning steps. In a few sentences, Diagnose a possible reason for failure and devise a new, concise, high level plan that aims to mitigate the same failure. Use complete sentences.  
Here are some examples:
{examples}

Previous trial:
Question: {question}{scratchpad}

Reflection:

エージェントの回答例

Question
Jaclyn Stapp is married to the former frontman of a band that disbanded in what year?
Answer
2004
Result
Question: Jaclyn Stapp is married to the former frontman of a band that disbanded in what year?
Thought 1: I need to search Jaclyn Stapp and the former frontman of a band, find the year the band disbanded, then find the answer.
Action 1: Search[Jaclyn Stapp]
Observation 1: Jaclyn Nesheiwat Stapp (born July 29, 1980) is a beauty queen, author, philanthropist, actress, and fashion model with pageant roots in Florida and New York. She is married to Scott Stapp, the voice of the band Creed and current solo touring artist.  Her most notable titles include Mrs. Florida America 2008, Miss New York USA 2004, and First Runner-Up for Mrs. World 2022. In 2010 she wrote a children's book, Wacky Jacky: The True Story of an Unlikely Beauty Queen.  She is executive director of The Scott Stapp With Arms Wide Open Foundation, and founder of CHARM (Children Are Magical) by Jaclyn Stapp, which was formed to raise awareness of issues with which children deal and to provide underprivileged youth help with education.
Thought 2: Jaclyn Stapp is married to Scott Stapp, the voice of the band Creed. I need to search Creed and find the year the band disbanded.
Action 2: Search[Creed]
Observation 2: A creed, also known as a confession of faith, a symbol, or a statement of faith, is a statement of the shared beliefs of a community (often a religious community) in a form which is structured by subjects which summarize its core tenets.
The earliest known creed in Christianity, "Jesus is Lord", originated in the writings of Paul the Apostle. One of the most widely used Christian creeds is the Nicene Creed, first formulated in AD 325 at the First Council of Nicaea. It was based on Christian understanding of the canonical gospels, the letters of the New Testament and, to a lesser extent, the Old Testament. Affirmation of this creed, which describes the Trinity, is generally taken as a fundamental test of orthodoxy for most Christian denominations, and was historically purposed against Arianism. A shorter version of the creed, called the Apostles' Creed, is nowadays the most used version in Christian services. 
Some Christian denominations do not use any of those creeds.
Although some say Judaism is non-creedal in nature, others say it recognizes a single creed, the Shema Yisrael, which begins: "Hear, O Israel: the LORD our God, the LORD is one."In Islamic theology, the term most closely corresponding to "creed" is ʿaqīdah (عقيدة).
Thought 3: Creed is not the band I am looking for. I need to search Creed (band) instead.
Action 3: Search[Creed (band)]
Observation 3: Creed was an American rock band from Tallahassee, Florida. Active from 1994 to 2004, and then from 2009 to 2012, Creed was prominent in the post-grunge movement of the mid-1990s. The band released three consecutive multi-platinum albums, with their album Human Clay being certified diamond. Creed has sold over 28 million records in the United States, has sold over 53 million albums worldwide, and was the ninth best-selling artist of the first decade of the 2000s. However, Creed has been panned by some critics and listeners; readers of Rolling Stone magazine ranked the band the worst artist of the 1990s.For most of its existence, the band consisted of lead vocalist Scott Stapp, guitarist and vocalist Mark Tremonti, bassist Brian Marshall, and drummer Scott Phillips. Creed released two studio albums, My Own Prison in 1997 and Human Clay in 1999, before Marshall left the band in 2000. The band's third album, Weathered, was released in 2001, with Tremonti on bass guitar. Creed disbanded in 2004; Stapp pursued a solo career while Tremonti, Marshall, and Phillips founded the band Alter Bridge with Myles Kennedy.
In 2009, Creed reunited for a fourth album, Full Circle, then toured until 2012. Since then, Creed has been on hiatus while the instrumental members have remained active with Alter Bridge; Stapp has continued his solo career and joined the band Art of Anarchy in 2016. Tremonti also formed his own band, Tremonti, in 2011.
Thought 4: Creed disbanded in 2004, so the answer is 2004.
Action 4: Finish[2004]
Observation 4: Answer is CORRECT
Correct answer: 2004

日本語訳

質問です： Jaclyn Stappは、何年に解散したバンドの元フロントマンと結婚している？
思考1：Jaclyn Stappとあるバンドの元フロントマンを検索して、そのバンドが解散した年を見つけ、それから答えを見つける必要がある。
行動1：[Jaclyn Stapp]を検索する。
観察1：Jaclyn Nesheiwat Stapp（1980年7月29日生まれ）は、フロリダとニューヨークのページェントをルーツとする美の女王、作家、慈善事業家、女優、ファッションモデル。バンド「クリード」の声優であり、現在はソロでツアーを行っているスコット・スタップと結婚している。 主な肩書きは、ミセス・フロリダ・アメリカ2008、ミス・ニューヨークUSA2004、ミセス・ワールド2022のファースト・ランナー・アップなど。2010年には児童書「Wacky Jacky: The True Story of an Unlikely Beauty Queen」を執筆した。 スコット・スタップ・ウィズ・アームズ・ワイド・オープン財団のエグゼクティブ・ディレクターであり、子供たちが抱える問題への意識を高め、恵まれない若者たちに教育支援を行うために結成されたCHARM（Children Are Magical）by Jaclyn Stappの創設者である。
思考2：Jaclyn Stappは、バンドCreedの声優であるScott Stappと結婚しています。クリードを検索して、バンドが解散した年を調べる必要がある。
行動2：[Creed]を検索する
観察2：信条とは、信仰告白、シンボル、信仰声明とも呼ばれ、ある共同体（多くは宗教共同体）が共有する信念を、その中核となる信条を要約した主題によって構成された形式で述べたものである。
キリスト教で最も古くから知られている信条は、「イエスは主である」というもので、使徒パウロの著作に由来しています。最も広く使われているキリスト教の信条は、AD325年の第1回ニカイア公会議で初めて制定された「ニカイア信条」である。この信条は、新約聖書の正典である福音書、手紙、および旧約聖書に対するキリスト教の理解に基づいている。三位一体について述べたこの信条を支持することは、一般に、ほとんどのキリスト教の教派にとって正統性の基本的なテストとされており、歴史的にアリウス主義に対する目的であった。この信条を短くしたものが「使徒信条」と呼ばれ、現在ではキリスト教の礼拝で最も多く使われています。
キリスト教の教派によっては、これらの信条を用いないものもある。
ユダヤ教はもともと信条を持たないという説もあるが、シェマ・イースレエルという一つの信条を認めているという説もある： 「イスラム神学では、「信条」に最も近い用語はʾaqīdah (عقيدة) である。
思考3：Creedは私が探しているバンドではありません。代わりにCreed (band)を検索する必要があります。
行動3：[Creed (band)]を検索する。
観察3：Creedは、フロリダ州タラハシー出身のアメリカのロックバンドである。1994年から2004年まで、そして2009年から2012年まで活動したクリードは、1990年代半ばのポスト・グランジ・ムーブメントにおいて著名な存在であった。バンドは3枚連続でマルチプラチナアルバムをリリースし、アルバム『Human Clay』はダイヤモンドに認定されました。クリードは米国で2800万枚以上のレコードを売り上げ、全世界で5300万枚以上のアルバムを売り上げ、2000年代の最初の10年間で9番目に売れたアーティストとなった。しかし、クリードは一部の批評家やリスナーから非難されている。ローリングストーン誌の読者は、バンドを1990年代の最悪のアーティストにランク付けした。バンドはその存在のほとんどで、リードボーカリストのスコット・スタップ、ギターとボーカルのマーク・トレモンティ、ベースのブライアン・マーシャル、ドラマーのスコット・フィリップスの4人で構成されている。クリードは1997年に『My Own Prison』、1999年に『Human Clay』という2枚のスタジオアルバムをリリースし、2000年にマーシャルがバンドを脱退した。2001年、トレモンティがベースを担当する3枚目のアルバム『Weathered』を発表。スタップはソロ活動を行い、トレモンティ、マーシャル、フィリップスはマイルス・ケネディと共にバンド「オルター・ブリッジ」を結成した後、2004年にクリードは解散した。
2009年、クリードは4枚目のアルバム『Full Circle』のために再結成し、2012年までツアーを行った。それ以降、Creedは活動休止しているが、楽器隊のメンバーはAlter Bridgeで活動を続けている。スタップはソロ活動を続け、2016年にバンドArt of Anarchyに加入した。また、トレモンティは2011年に自身のバンド「Tremonti」を結成している。
思考4：Creedは2004年に解散しているので、答えは2004年です。
行動4：フィニッシュ[2004]。
観察4：答えは「CORRECT」です。
正解：2004年

「Creed」という単語で検索してしまい一般名詞の説明が返ってきた結果を踏まえて、「Creed(band)」という検索に修正しているのが印象的です。

プロンプトに埋め込まれるexamples

react_reflect_agent_promptのexamples

Question: What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into?
Thought 1: I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area.
Action 1: Search[Colorado orogeny]
Observation 1: The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas.
Thought 2: It does not mention the eastern sector. So I need to look up eastern sector.
Action 2: Lookup[eastern sector]
Observation 2: (Result 1 / 1) The eastern sector extends into the High Plains and is called the Central Plains orogeny.
Thought 3: The eastern sector of Colorado orogeny extends into the High Plains. So I need to search High Plains and find its elevation range.
Action 3: Search[High Plains]
Observation 3: High Plains refers to one of two distinct land regions:
Thought 4: I need to instead search High Plains (United States).
Action 4: Search[High Plains (United States)]
Observation 4: The High Plains are a subregion of the Great Plains. From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130 m).[3]
Thought 5: High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft.
Action 5: Finish[1,800 to 7,000 ft]

Question: Musician and satirist Allie Goertz wrote a song about the "The Simpsons" character Milhouse, who Matt Groening named after who?
Thought 1: The question simplifies to "The Simpsons" character Milhouse is named after who. I only need to search Milhouse and find who it is named after.
Action 1: Search[Milhouse]
Observation 1: Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening.
Thought 2: The paragraph does not tell who Milhouse is named after, maybe I can look up "named after".
Action 2: Lookup[named after]
Observation 2: (Result 1 / 1) Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous. 
Thought 3: Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon.
Action 3: Finish[Richard Nixon]

Question: Which documentary is about Finnish rock groups, Adam Clayton Powell or The Saimaa Gesture?
Thought 1: I need to search Adam Clayton Powell and The Saimaa Gesture, and find which documentary is about Finnish rock groups.
Action 1: Search[Adam Clayton Powell]
Observation 1: Could not find [Adam Clayton Powell]. Similar: ['Adam Clayton Powell III', 'Seventh Avenue (Manhattan)', 'Adam Clayton Powell Jr. State Office Building', 'Isabel Washington Powell', 'Adam Powell', 'Adam Clayton Powell (film)', 'Giancarlo Esposito'].
Thought 2: To find the documentary, I can search Adam Clayton Powell (film).
Action 2: Search[Adam Clayton Powell (film)]
Observation 2: Adam Clayton Powell is a 1989 American documentary film directed by Richard Kilberg.
The film is about the rise and fall of influential African-American politician Adam Clayton Powell Jr.[3][4] It was later aired as part of the PBS series The American Experience.
Thought 3: Adam Clayton Powell (film) is a documentary about an African-American politician, not Finnish rock groups. So the documentary about Finnish rock groups must instead be The Saimaa Gesture.
Action 3: Finish[The Saimaa Gesture]

Question: What profession does Nicholas Ray and Elia Kazan have in common?
Thought 1: I need to search Nicholas Ray and Elia Kazan, find their professions, then find the profession they have in common.
Action 1: Search[Nicholas Ray]
Observation 1: Nicholas Ray (born Raymond Nicholas Kienzle Jr., August 7, 1911 – June 16, 1979) was an American film director, screenwriter, and actor best known for the 1955 film Rebel Without a Cause.
Thought 2: Professions of Nicholas Ray are director, screenwriter, and actor. I need to search Elia Kazan next and find his professions.
Action 2: Search[Elia Kazan]
Observation 2: Elia Kazan was an American film and theatre director, producer, screenwriter and actor.
Thought 3: Professions of Elia Kazan are director, producer, screenwriter, and actor. So profession Nicholas Ray and Elia Kazan have in common is director, screenwriter, and actor.
Action 3: Finish[director, screenwriter, actor]

Question: Which magazine was started first Arthur's Magazine or First for Women?
Thought 1: I need to search Arthur's Magazine and First for Women, and find which was started first.
Action 1: Search[Arthur's Magazine]
Observation 1: Arthur's Magazine (1844-1846) was an American literary periodical published in Philadelphia in the 19th century. 
Thought 2: Arthur's Magazine was started in 1844. I need to search First for Women next.
Action 2: Search[First for Women]
Observation 2: First for Women is a woman's magazine published by Bauer Media Group in the USA.[1] The magazine was started in 1989. 
Thought 3: First for Women was started in 1989. 1844 (Arthur's Magazine) < 1989 (First for Women), so Arthur's Magazine was started first.
Action 3: Finish[Arthur's Magazine]

Question: Were Pavel Urysohn and Leonid Levin known for the same type of work?
Thought 1: I need to search Pavel Urysohn and Leonid Levin, find their types of work, then find if they are the same.
Action 1: Search[Pavel Urysohn]
Observation 1: Pavel Samuilovich Urysohn (February 3, 1898 â August 17, 1924) was a Soviet mathematician who is best known for his contributions in dimension theory.
Thought 2: Pavel Urysohn is a mathematician. I need to search Leonid Levin next and find its type of work.
Action 2: Search[Leonid Levin]
Observation 2: Leonid Anatolievich Levin is a Soviet-American mathematician and computer scientist. 
Thought 3: Leonid Levin is a mathematician and computer scientist. So Pavel Urysohn and Leonid Levin have the same type of work. 
Action 3: Finish[yes]

reflect_promptのexamples

Previous Trial:
Question: The Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of what?
Thought 1: I need to search Rome Protocols, find the three Prime Ministers, then find what they were assassinated as part of.
Action 1: Search[Rome Protocols]
Observation 1: The Rome Protocols were a series of three international agreements concluded in Rome on 17 March 1934 between the governments of Austria, Hungary and Italy. They were signed by Italian Prime Minister Benito Mussolini, Austrian Chancellor Engelbert Dollfuss and Hungarian Prime Minister Gyula Gömbös. All of the protocols went into effect on 12 July 1934 and were registered in League of Nations Treaty Series on 12 December 1934.
Thought 2: The three Prime Ministers were Benito Mussolini, Engelbert Dollfuss, and Gyula Gömbös. I need to search Benito Mussolini and find what he was assassinated as part of.
Action 2: Search[Benito Mussolini]
Observation 2: Benito Amilcare Andrea Mussolini (UK:  MU(U)SS-ə-LEE-nee, US:  MOOSS-, Italian: [beˈniːto aˈmilkare anˈdrɛːa mussoˈliːni]; 29 July 1883 – 28 April 1945) was an Italian politician and journalist who founded and led the National Fascist Party (PNF). He was Prime Minister of Italy from the March on Rome in 1922 until his deposition in 1943, as well as "Duce" of Italian fascism from the establishment of the Italian Fasces of Combat in 1919 until his summary execution in 1945 by Italian partisans. As dictator of Italy and principal founder of fascism, Mussolini inspired and supported the international spread of fascist movements during the inter-war period.Mussolini was originally a socialist politician and a journalist at the Avanti! newspaper. In 1912, he became a member of the National Directorate of the Italian Socialist Party (PSI), but he was expelled from the PSI for advocating military intervention in World War I, in opposition to the party's stance on neutrality. In 1914, Mussolini founded a new journal, Il Popolo d'Italia, and served in the Royal Italian Army during the war until he was wounded and discharged in 1917. Mussolini denounced the PSI, his views now centering on Italian nationalism instead of socialism, and later founded the fascist movement which came to oppose egalitarianism and class conflict, instead advocating "revolutionary nationalism" transcending class lines. On 31 October 1922, following the March on Rome (28–30 October), Mussolini was appointed prime minister by King Victor Emmanuel III, becoming the youngest individual to hold the office up to that time. After removing all political opposition through his secret police and outlawing labor strikes, Mussolini and his followers consolidated power through a series of laws that transformed the nation into a one-party dictatorship. Within five years, Mussolini had established dictatorial authority by both legal and illegal means and aspired to create a totalitarian state. In 1929, Mussolini signed the Lateran Treaty with the Holy See to establish Vatican City.
Mussolini's foreign policy aimed to restore the ancient grandeur of the Roman Empire by expanding Italian colonial possessions and the fascist sphere of influence. In the 1920s, he ordered the Pacification of Libya, instructed the bombing of Corfu over an incident with Greece, established a protectorate over Albania, and incorporated the city of Fiume into the Italian state via agreements with Yugoslavia. In 1936, Ethiopia was conquered following the Second Italo-Ethiopian War and merged into Italian East Africa (AOI) with Eritrea and Somalia. In 1939, Italian forces annexed Albania. Between 1936 and 1939, Mussolini ordered the successful Italian military intervention in Spain in favor of Francisco Franco during the Spanish Civil War. Mussolini's Italy initially tried to avoid the outbreak of a second global war, sending troops at the Brenner Pass to delay Anschluss and taking part in the Stresa Front, the Lytton Report, the Treaty of Lausanne, the Four-Power Pact and the Munich Agreement. However, Italy then alienated itself from Britain and France by aligning with Germany and Japan. Germany invaded Poland on 1 September 1939, resulting in declarations of war by France and the UK and the start of World War II.
On 10 June 1940, Mussolini decided to enter the war on the Axis side. Despite initial success, the subsequent Axis collapse on multiple fronts and eventual Allied invasion of Sicily made Mussolini lose the support of the population and members of the Fascist Party. As a consequence, early on 25 July 1943, the Grand Council of Fascism passed a motion of no confidence in Mussolini; later that day King Victor Emmanuel III dismissed him as head of government and had him placed in custody, appointing Pietro Badoglio to succeed him as Prime Minister. After the king agreed to an armistice with the Allies, on 12 September 1943 Mussolini was rescued from captivity in the Gran Sasso raid by German paratroopers and Waffen-SS commandos led by Major Otto-Harald Mors. Adolf Hitler, after meeting with the rescued former dictator, then put Mussolini in charge of a puppet regime in northern Italy, the Italian Social Republic (Italian: Repubblica Sociale Italiana, RSI), informally known as the Salò Republic, causing a civil war. In late April 1945, in the wake of near total defeat, Mussolini and his mistress Clara Petacci attempted to flee to Switzerland, but both were captured by Italian communist partisans and summarily executed by firing squad on 28 April 1945 near Lake Como. The bodies of Mussolini and his mistress were then taken to Milan, where they were hung upside down at a service station to publicly confirm their demise.
Thought 3: Benito Mussolini was assassinated as part of the collapse of the Axis powers during World War II. So the Rome Protocols were signed by three Prime Ministers one of which was assassinated as part of World War II.
Action 3: Finish[World War II]

Reflection: I searched one of the prime ministers involved in the signing, then attemted to answer right away. I should have searched each of the prime ministers, then looked up 'death' on each of their pages in order to get more information before answering.

Previous Trial:
Question: Kam Heskin plays Paige Morgan in a 2004 film directed by who?
Thought 1: I need to search Kam Heskin and Paige Morgan, find the 2004 film they are in, then find the director of the film.
Action 1: Search[Kam Heskin]
Observation 1: Kam Erika Heskin (born May 8, 1973) is an American actress. She began her career playing Caitlin Deschanel on the NBC daytime soap opera Sunset Beach (1998–1999), before appearing in films Planet of the Apes (2001) and Catch Me If You Can (2002). Heskin went on in 2003 to play Elizabeth Bennet in an independent film Pride & Prejudice: A Latter-Day Comedy, Robin in Charmed (2004),  and Paige Morgan in The Prince and Me film franchise (2006–2010).
Thought 2: Kam Heskin plays Paige Morgan in The Prince and Me film franchise. I need to search The Prince and Me film franchise and find the director of the 2004 film.
Action 2: Search[The Prince and Me film franchise]
Observation 2: Could not find [The Prince and Me film franchise]. Similar: ['Kam Heskin', 'Prince of Persia', 'Harry Potter and the Half-Blood Prince (film)', 'Prince of Persia: The Sands of Time (film)', 'The Chronicles of Narnia (film series)', 'Majestic Prince (franchise)', 'Kingsman (franchise)', 'The Little Prince (2015 film)', 'Shrek (franchise)', 'Harry Potter (film series)']
Thought 3: I need to search The Prince and Me (2004 film) instead.
Action 3: Search[The Prince and Me (2004 film)]
Observation 3: Could not find [The Prince and Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Kam Heskin', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince of Persia', 'Brooklynn Prince', 'Prince (musician)']
Thought 4: I need to search The Prince & Me (2004 film).
Action 4: Search[The Prince & Me (2004 film)]
Observation 4: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince']
Thought 5: I need to search The Prince & Me (2004 film) instead.
Action 5: Search[The Prince & Me (2004 film)]
Observation 5: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince']
Thought 6: I need to search The Prince & Me (2004 film).
Action 6: Search[The Prince & Me (2004 film)]
Observation 6: Could not find [The Prince & Me (2004 film)]. Similar: ['The Prince & Me', 'The Prince & Me 2: The Royal Wedding', 'Prince of Darkness (film)', 'Prince of Persia: The Sands of Time (film)', 'Rob Knox', 'Alexander (2004 film)', 'Prince (musician)', 'Prince of Persia', 'Kam Heskin', 'Brooklynn Prince']

Reflection: I got stuck in a loop where I kept trying to search 'The Prince & Me (2004 film)' but the page could not be found. Instead I should have tried to search the similar results that had a similar name to see and they were made in 2004.