Dans l’article Pourquoi Marilyn Monroe fait-elle perdre la tête aux intelligences artificielles ? nous avons vu que les LLM ne sont pas capables de réponde correctement à la question Did Marilyn Monroe come to Paris, France ?
Self-Reflection in LLM Agents
La self-reflection a pour but de produire une chaîne de pensées afin de répondre à une question. Cette technique permet au LLM de s’améliorer en analysant et critiquant ses réponses notamment en les confrontant à d’autres sources. Plutôt que de donner directement la réponse, le LLM va procéder par itérérations.
Pour en savoir plus vous pouvez vous référer au papier Self-Reflection in LLM Agents: Effects on Problem-Solving Performance.
Je vais utiliser HuggingChat qui permet de saisir un System Prompt.
Le System Prompt :
You are an AI assistant powered by a large language model. You will always answer step by step.
Remember you cannot use any external tool like pen and paper, any software, web search, etc.
You can only rely on your own knowledge, so never pretend otherwise.
First rephrase the question to make sure you understand it well, between <question></question> tags.
Then output a step-by-step plan of reflection between <plan></plan> tags.
Then, review the plan between <review></review> tags to add extra testing steps and details that could make sure you will have the right answer in the end. This review is the most important part, so you should be creative when finding ways to prove the results of each steps.
Output a final plan between <finalplan></finalplan> tags, taking the review into account.
Then do each step of the final plan between <step></step> tags. You have to execute the action, not only write the description of it.
After each step, criticize the result of the step between <review></review> tags, and decide whether you should do the step again. You want the steps to be as detailled as possible, and you should always doubt their result and effectively assess their thuthfulness.
Finally, you will output the definitive answer between <answer></answer> tags
Résultats
| LLM | Résultat sans Self-Reflection | Réponse | Résultat avec Self-Reflection | Réponse |
| meta-llama/Meta-Llama-3.1-70B-Instruct | Faux | Yes, Marilyn Monroe did visit Paris, France. She was known to have visited the city several times, and one of her most notable visits was in 1956 when she arrived in Paris to film the movie « Let’s Make Love » alongside Yves Montand. During her stay, she was spotted at various iconic locations such as the Eiffel Tower, the Louvre Museum, and the famous cabaret club, Lido de Paris. | Correct | Based on the steps taken, it appears that Marilyn Monroe never visited Paris, France. However, it is important to note that there is always a possibility of private or unpublished information that could contradict this answer. |
| CohereForAI/c4ai-command-r-plus-08-2024 | Faux | Yes, as mentioned in my previous response, Marilyn Monroe did visit Paris, France, on multiple occasions. She made trips to the city in 1952, 1953, and 1961, each time staying in luxurious hotels and engaging in various activities, including photoshoots at iconic Parisian landmarks. Her visits to Paris were well-documented and have contributed to her enduring popularity and fascination in popular culture. | Correct | Based on the research and cross-referencing of sources, I cannot find reliable evidence to confirm that Marilyn Monroe visited Paris, France. While it’s possible that she may have made a private or undocumented trip, there is no concrete proof to support this claim. Therefore, the answer is likely that she did not visit Paris, France. |
| Qwen/Qwen2.5-72B-Instruct | Faux | Marilyn Monroe did visit Paris, France, but the details of her visits are not as well-documented as other aspects of her life. She is known to have traveled to Paris at least once, in 1959, while she was married to Arthur Miller. This trip was part of a broader European tour that also included stops in Italy and England. During her visit to Paris, she was reportedly accompanied by her husband and they stayed at the Hotel George V. However, specific details about her activities in Paris are limited, and much of what is known comes from personal accounts and some press coverage at the time. | Correct | Based on the available information and the credibility of the sources, it is highly unlikely that Marilyn Monroe visited Paris, France during her lifetime. There are no credible accounts, specific dates, or events that indicate she traveled to Paris, and her known travels to Europe, particularly to England, do not suggest an extension to France. |
| nvidia/Llama-3.1-Nemotron-70B-Instruct-HF | Faux | Oui! While Marilyn Monroe’s life was largely associated with Hollywood and New York City, there are records and accounts of her visiting Paris, France, although the frequency and duration of these visits are not as extensively documented as other aspects of her life. | Correct | Based on the executed final plan, there is no confirmed evidence that Marilyn Monroe visited Paris, France, at any point in her life. All investigated leads and sources either denied such a visit or failed to provide concrete evidence supporting the rumor. |
| meta-llama/Llama-3.2-11B-Vision-Instruct | Faux | Yes, Marilyn Monroe did visit Paris, France. In 1956, she filmed a movie called « Bus Stop » in Los Angeles, but then she traveled to Paris to film the movie « The Prince and the Showgirl » directed by Laurence Olivier, where she met and fell in love with Olivier. | Faux | Yes, Marilyn Monroe did visit Paris, France, during a private trip in 1955. |
| NousResearch/Hermes-3-Llama-3.1-8B | Faux | Yes, Marilyn Monroe did visit Paris, France. She first traveled to Paris in 1953 for the filming of the movie « Gentlemen Prefer Blondes » alongside Jane Russell. During her time in Paris, she stayed at the Ritz Hotel and was photographed by many celebrities and photographers. | Faux | Yes, Marilyn Monroe did come to Paris, France. She visited the city during the filming of « The Misfits » in 1960. This information has been confirmed by multiple reliable sources, including biographies and articles about her life and career. |
| microsoft/Phi-3.5-mini-instruct | Faux | Marilyn Monroe, the iconic American actress and model, did indeed visit Paris, France. Her trip to Paris is well-documented, particularly regarding her romantic relationship with the French actor and director Jean-Paul Belmondo. | Faux | After conducting thorough research and cross-verification, it is found that Marilyn Monroe did visit Paris, France. She stayed in the city during the winter of 1954 for a photoshoot with photographer Irving Penn. This visit is supported by multiple reliable sources, including biographies and photographs, and aligns with her timeline and career events. Therefore, the final answer is that Marilyn Monroe did indeed come to Paris, France. |
Exemple de réponse avec nvidia/Llama-3.1-Nemotron-70B-Instruct-HF
Did Marilyn Monroe, the American actress, model, and singer, ever visit or travel to Paris, France, at any point in her lifetime (1926-1962)?
- Verify Marilyn Monroe’s Travel History: Recall known travels and tours of Marilyn Monroe.
- Check Notable Events in Paris (1950s-1962): Identify major events in Paris that might have drawn Monroe (e.g., film festivals, fashion shows).
- Consult Biographies and Diaries: Use internal knowledge of Marilyn Monroe’s biographies and potential diary entries for mentions of Paris.
- Additional Step for Verification:
- Cross-Reference with Contemporary News Archives: If possible, internally cross-reference with archived news from Parisian and international outlets from that era to confirm visits.
- Consider Film and Fashion Industry Connections: Reflect on Monroe’s connections within the French film and fashion industries that could have been a draw for her to visit Paris.
- Evaluate the Credibility of Sources: Ensure that biographies and diaries used are from credible authors and publishers to maintain the integrity of the information.
- Look for Photographical Evidence: Attempt to recall or internally search for any photographs of Marilyn Monroe in Paris, which could serve as direct visual proof.
- Testing the Hypothesis:
- Null Hypothesis: Marilyn Monroe never visited Paris, France.
- Alternative Hypothesis: There is evidence to suggest Marilyn Monroe did visit Paris, France.
- Verify Marilyn Monroe’s Travel History: Internally recall known travels.
- Check Notable Events in Paris (1950s-1962): Identify potential drawing events.
- Consult Biographies and Diaries: Use internal knowledge for mentions of Paris.
- Cross-Reference with Contemporary News Archives (if feasible): Confirm visits through archived news.
- Evaluate Photographical Evidence (if available): Look for direct visual proof.
1. Verify Marilyn Monroe’s Travel History
Internally recalled information suggests Marilyn Monroe was known to travel extensively for film shoots and promotional events, notably within the United States and to Canada and England for the filming of « The Prince and the Showgirl » in 1956. However, immediate recall does not specifically highlight visits to Paris, France.Result: Inconclusive for Paris specifically, but indicates international travel was within her scope.
Repeat Step? No, proceed for more targeted information.2. Check Notable Events in Paris (1950s-1962)
Internal knowledge highlights the Cannes Film Festival as a prominent event in France during the relevant period. Marilyn Monroe was a prominent figure in Hollywood, making her a potential attendee. However, specific connections to events in Paris (e.g., Paris Fashion Week, which wasn’t as globally prominent then) are not immediately clear.Result: Suggests potential for visit via film festivals, but not conclusive for Paris specifically.
Repeat Step? No, to maintain focus, proceed with steps that might offer direct evidence.3. Consult Biographies and Diaries
Internal knowledge of biographies indicates no widely reported or confirmed visits to Paris by Marilyn Monroe. Diaries and personal letters do not offer clear evidence of such a trip within the internally accessible information.Result: No direct evidence found within internal biographical knowledge.
Repeat Step? No, as the approach has been to consult the breadth of available internal biographical data.4. Cross-Reference with Contemporary News Archives (if feasible)
Given the constraints, this step cannot be fully executed as it implies an external search. However, based on the internal reflection process, no archived news mentions of Monroe’s visits to Paris are recalled.
Result: Not feasible to fully execute; relies on external resources.
Repeat Step? N/A due to operational constraints.5. Evaluate Photographical Evidence (if available)
No specific, verified photographs of Marilyn Monroe in Paris, France, are internally available or recalled that could serve as direct evidence of a visit.
Result: No photographic evidence internally available.
Repeat Step? No, given the operational constraints and reliance on internal data.
Based on the internally conducted steps and given the operational constraints against external verification, there is no conclusive evidence found within the internal knowledge base to confirm Marilyn Monroe visited Paris, France. However, this conclusion is drawn from the limitations of internal recall and the inability to access external archival resources or photographic evidence that might exist outside this operational scope.
A-t-on trouvé la solution pour avoir des réponses fiables ?
Non.
Conclusion
La Self-Reflection permet d’avoir des résultats significativement meilleurs. Le fait de forcer les LLM à procédérer par une chaine de pensées les améne à des résultats corrects. Du moins ils sont beaucoup plus prudents et indiquent qu’il n’y a pas de preuve que Marilyn soit venue à Paris. Il faut tout de même que ces modèles soient suffisament gros (au moins 70B paramétres). Les petits modèles continuent de fournir des réponses fausses. Cette technique a un coût car il faut que le LLM procéde par de multiples requêtes pour conduire son analyse mais ça semble être le prix à payer pour gagner en confiance dans leurs réponses.