Episode Analysis: Yes (GT: 1)
Query Descriptions (What We Search For)
- a wooden muug with a white text and a small figure on it
- a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
- a sloth - faced wooden mug with a heart and a white writing on it
Target in Image (Ground Truth)
- a wooden muug with a white text and a small figure on it
- a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
- a sloth - faced wooden mug with a heart and a white writing on it
Step 1
Decision: Unsure
/root/autodl-tmp/pv_dataset/pin_capture/val/00876-mv2HUxq3B53/35/rgb/rgb_s6_far.png
Coarse Category
mug (Cache)
Navigation
front-left (MLLM-Reasoning)
NBV Reasoning (LLM)
Prompt:
You are an embodied perception planner for attribute-based object verification.
**Task**: The following 3 sentences ALL describe the SAME target object. We have decomposed these descriptions into specific attributes to verify.
**Object Description (3 sentences about the SAME object)**:
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
**Object Category**: mug
**Attribute Verification Status**:
[VERIFIED] material: 'wooden'
[CONTRADICTED] color: expected 'white'
[UNVERIFIED] logo_text: 'i'll do it tomorrow'
[CONTRADICTED] relation_1: expected 'a sloth hanged on the rim'
[UNVERIFIED] relation_2: 'a pink heart between the arms'
[UNVERIFIED] relation_3: 'a leaf near the handle'
[CONTRADICTED] part_attribute: expected 'a sloth - faced wooden mug'
**Navigation State**:
- Current View: You are currently viewing the object from the FRONT
- Already Visited: front
- **Available Directions to Move**: ['front-left', 'front-right', 'back', 'back-left', 'back-right']
**Direction Guide** (all directions are relative to YOUR current position):
- front-left: move to YOUR left side (you will see the object from a left angle)
- front-right: move to YOUR right side (you will see the object from a right angle)
- back: move to the opposite side of the object
- back-left: move to the left side of the opposite view
- back-right: move to the right side of the opposite view
**IMPORTANT**: You MUST choose ONE direction STRICTLY from the **Available Directions to Move** list above.
- Do NOT choose a direction that is NOT in the list.
- Do NOT choose 'front'; it means "stay in place" and is never a valid navigation direction.
- Do NOT choose a direction that is listed as "Already Visited".
**Your Task**: Analyze each available direction, then choose the best one.
**STEP 1 — Per-Direction Analysis** (you MUST analyze ALL available directions):
For EACH direction in ['front-left', 'front-right', 'back', 'back-left', 'back-right'], assess:
a) **Reachability**: Look at the scene around the object. Is the path in that direction blocked by walls, furniture, or other obstacles? Is there enough space to move there?
b) **Target Visibility**: If you move there, will you be able to see the target object, or might it be occluded by obstacles or the object's own geometry?
c) **Information Gain**: Which missing/unverified attributes could you potentially observe from that angle?
**STEP 2 — Direction Decision**:
Based on your Step 1 analysis, choose the direction that:
1. Is most likely reachable (clear path, no obstacles)
2. Will likely give you a clear view of the target object
3. Has the highest potential to reveal missing/unverified attributes
Output EXACTLY this JSON format:
{"per_direction_analysis": "<your analysis of each available direction>", "chosen_direction": "<must be from Available Directions>", "why": "<brief reason linking to missing attributes and reachability>"}
Response:
{
"per_direction_analysis": "front-left: Reachable, as there is open space to the left of the table. Target visibility is clear; the mug is centered and not occluded. Information gain: may reveal the sloth figure and text from a side angle. front-right: Reachable, with open space to the right. Target visibility is clear. Information gain: may reveal the sloth figure and text from a side angle. back: Reachable, as the table is in an open area. Target visibility is clear; the mug is centered. Information gain: may reveal the sloth figure and text from a rear angle, but less likely to show details like the heart or leaf. back-left: Reachable, with open space. Target visibility is clear. Information gain: may reveal the sloth figure and text from a rear-left angle, but less likely to show details like the heart or leaf. back-right: Reachable, with open space. Target visibility is clear. Information gain: may reveal the sloth figure and text from a rear-right angle, but less likely to show details like the heart or leaf.",
"chosen_direction": "front-left",
"why": "front-left offers the best balance of reachability, clear visibility, and potential to observe unverified attributes like the sloth figure and text from a side angle, which are critical for verifying the object's identity as described."
}Model Input (Crop/Ref)
Attribute Verification
| Attribute | Expected | Step Result | Tracker |
|---|---|---|---|
| material | wooden | Matched | Matched |
| color | white | Contradictory | Contradictory |
| logo_text | i'll do it tomorrow | Missing | Missing |
| relation_1 | a sloth hanged on the rim | Contradictory | Contradictory |
| relation_2 | a pink heart between the arms | Missing | Missing |
| relation_3 | a leaf near the handle | Missing | Missing |
| part_attribute | a sloth - faced wooden mug | Contradictory | Contradictory |
Verification Details (7 items)
Target: material (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
material = wooden
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object in the image is a mug, and its material is clearly wooden, as indicated by the visible wood grain texture."
}Reason: The object in the image is a mug, and its material is clearly wooden, as indicated by the visible wood grain texture.
Target: color (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
color = white
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is a wooden container with a white substance inside, but it is not a mug. The description mentions a sloth and text, which are not visible in the image, and the object's category does not match 'mug'."
}Reason: The object in the image is a wooden container with a white substance inside, but it is not a mug. The description mentions a sloth and text, which are not visible in the image, and the object's category does not match 'mug'.
Target: logo_text (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
logo_text = i'll do it tomorrow
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image shows a wooden mug with a white substance (possibly foam or ice) and a green straw, but no visible text or logo. The attribute 'logo_text = i'll do it tomorrow' cannot be verified from the image as it is not legible or present."
}Reason: The image shows a wooden mug with a white substance (possibly foam or ice) and a green straw, but no visible text or logo. The attribute 'logo_text = i'll do it tomorrow' cannot be verified from the image as it is not legible or present.
Target: relation_1 (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_1 = a sloth hanged on the rim
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is a wooden container with a white substance inside, not a mug with a sloth or text as described. It does not match the category 'mug' as defined by the object's visual characteristics."
}Reason: The object in the image is a wooden container with a white substance inside, not a mug with a sloth or text as described. It does not match the category 'mug' as defined by the object's visual characteristics.
Target: relation_2 (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_2 = a pink heart between the arms
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image shows a wooden mug with a white substance (possibly foam or ice) and a green straw, but no sloth, pink heart, or visible text. The attribute 'a pink heart between the arms' is not visible or verifiable in the image."
}Reason: The image shows a wooden mug with a white substance (possibly foam or ice) and a green straw, but no sloth, pink heart, or visible text. The attribute 'a pink heart between the arms' is not visible or verifiable in the image.
Target: relation_3 (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_3 = a leaf near the handle
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image shows a wooden mug with a white substance (possibly foam or ice) and a green leaf-like object near the top, but it is unclear whether this is a 'leaf near the handle' as described. The leaf's position and relation to the handle are ambiguous due to the image's resolution and angle."
}Reason: The image shows a wooden mug with a white substance (possibly foam or ice) and a green leaf-like object near the top, but it is unclear whether this is a 'leaf near the handle' as described. The leaf's position and relation to the handle are ambiguous due to the image's resolution and angle.
Target: part_attribute (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
part_attribute = a sloth - faced wooden mug
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is a wooden container with a white substance inside, not a mug with a sloth face or any of the described attributes. It does not match the category 'mug' as defined in the prompt."
}Reason: The object in the image is a wooden container with a white substance inside, not a mug with a sloth face or any of the described attributes. It does not match the category 'mug' as defined in the prompt.
Step 2
Decision: Unsure
/root/autodl-tmp/pv_dataset/pin_capture/val/00876-mv2HUxq3B53/35/rgb/rgb_s8_far.png
Coarse Category
mug (Cache)
Navigation
front-left (MLLM-Reasoning)
NBV Reasoning (LLM)
Prompt:
You are an embodied perception planner for attribute-based object verification.
**Task**: The following 3 sentences ALL describe the SAME target object. We have decomposed these descriptions into specific attributes to verify.
**Object Description (3 sentences about the SAME object)**:
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
**Object Category**: mug
**Attribute Verification Status**:
[VERIFIED] material: 'wooden'
[CONTRADICTED] color: expected 'white'
[UNVERIFIED] logo_text: 'i'll do it tomorrow'
[CONTRADICTED] relation_1: expected 'a sloth hanged on the rim'
[UNVERIFIED] relation_2: 'a pink heart between the arms'
[UNVERIFIED] relation_3: 'a leaf near the handle'
[CONTRADICTED] part_attribute: expected 'a sloth - faced wooden mug'
**Navigation State**:
- Current View: You are currently viewing the object from the FRONT
- Already Visited: front-right, front
- **Available Directions to Move**: ['front-left', 'back', 'back-left', 'back-right']
**Direction Guide** (all directions are relative to YOUR current position):
- front-left: move to YOUR left side (you will see the object from a left angle)
- back: move to the opposite side of the object
- back-left: move to the left side of the opposite view
- back-right: move to the right side of the opposite view
**IMPORTANT**: You MUST choose ONE direction STRICTLY from the **Available Directions to Move** list above.
- Do NOT choose a direction that is NOT in the list.
- Do NOT choose 'front'; it means "stay in place" and is never a valid navigation direction.
- Do NOT choose a direction that is listed as "Already Visited".
**Your Task**: Analyze each available direction, then choose the best one.
**STEP 1 — Per-Direction Analysis** (you MUST analyze ALL available directions):
For EACH direction in ['front-left', 'back', 'back-left', 'back-right'], assess:
a) **Reachability**: Look at the scene around the object. Is the path in that direction blocked by walls, furniture, or other obstacles? Is there enough space to move there?
b) **Target Visibility**: If you move there, will you be able to see the target object, or might it be occluded by obstacles or the object's own geometry?
c) **Information Gain**: Which missing/unverified attributes could you potentially observe from that angle?
**STEP 2 — Direction Decision**:
Based on your Step 1 analysis, choose the direction that:
1. Is most likely reachable (clear path, no obstacles)
2. Will likely give you a clear view of the target object
3. Has the highest potential to reveal missing/unverified attributes
Output EXACTLY this JSON format:
{"per_direction_analysis": "<your analysis of each available direction>", "chosen_direction": "<must be from Available Directions>", "why": "<brief reason linking to missing attributes and reachability>"}
Response:
{
"per_direction_analysis": "front-left: The path is clear, with no furniture blocking the way. From this angle, the mug's front-left side may reveal the sloth figure and any text or heart near the rim. back: The path is clear, but the object will be seen from the rear, which may obscure the sloth, text, and heart details. back-left: The path is clear, but the angle may be obstructed by the surrounding furniture, limiting visibility of the mug's features. back-right: The path is clear, but the angle may show only the side of the mug, potentially missing the sloth and text details.",
"chosen_direction": "front-left",
"why": "The front-left direction offers the best visibility of the sloth figure, text, and heart, which are unverified attributes, while maintaining clear reachability and minimal obstruction."
}Model Input (Crop/Ref)
Attribute Verification
| Attribute | Expected | Step Result | Tracker |
|---|---|---|---|
| material | wooden | Matched | Matched |
| color | white | Missing | Contradictory |
| logo_text | i'll do it tomorrow | Missing | Missing |
| relation_1 | a sloth hanged on the rim | Contradictory | Contradictory |
| relation_2 | a pink heart between the arms | Contradictory | Missing |
| relation_3 | a leaf near the handle | Missing | Missing |
| part_attribute | a sloth - faced wooden mug | Contradictory | Contradictory |
Verification Details (7 items)
Target: material (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
material = wooden
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object in the image is a mug, and its material is clearly wooden, as indicated by the visible wood grain texture."
}Reason: The object in the image is a mug, and its material is clearly wooden, as indicated by the visible wood grain texture.
Target: color (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
color = white
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image shows a wooden mug with a white interior or rim, but the visible exterior color is brown. The attribute 'color = white' refers to the entire object, which is not clearly white — only part of it (interior or rim) appears white. Therefore, we cannot confirm the full object color is white."
}Reason: The image shows a wooden mug with a white interior or rim, but the visible exterior color is brown. The attribute 'color = white' refers to the entire object, which is not clearly white — only part of it (interior or rim) appears white. Therefore, we cannot confirm the full object color is white.
Target: logo_text (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
logo_text = i'll do it tomorrow
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image shows a wooden mug with a white interior and a handle, but the text 'i'll do it tomorrow' is not visible or legible in the provided image. Therefore, we cannot verify the attribute."
}Reason: The image shows a wooden mug with a white interior and a handle, but the text 'i'll do it tomorrow' is not visible or legible in the provided image. Therefore, we cannot verify the attribute.
Target: relation_1 (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_1 = a sloth hanged on the rim
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is a wooden container or bucket, not a mug. It lacks the typical mug shape with a handle and rim, and no sloth or heart is visible."
}Reason: The object in the image is a wooden container or bucket, not a mug. It lacks the typical mug shape with a handle and rim, and no sloth or heart is visible.
Target: relation_2 (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_2 = a pink heart between the arms
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is a wooden container or bucket, not a mug. It lacks the typical mug shape with a handle and rim, and no sloth, heart, or text is visible."
}Reason: The object in the image is a wooden container or bucket, not a mug. It lacks the typical mug shape with a handle and rim, and no sloth, heart, or text is visible.
Target: relation_3 (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_3 = a leaf near the handle
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image shows a wooden container with a white interior and a handle, but no leaf is visible near the handle. The object's exact features are not clear enough to confirm the presence of a leaf."
}Reason: The image shows a wooden container with a white interior and a handle, but no leaf is visible near the handle. The object's exact features are not clear enough to confirm the presence of a leaf.
Target: part_attribute (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
part_attribute = a sloth - faced wooden mug
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is a wooden container or bucket, not a mug. It lacks the characteristic shape of a mug (e.g., handle, rim for drinking) and does not match the description of a 'sloth-faced wooden mug'."
}Reason: The object in the image is a wooden container or bucket, not a mug. It lacks the characteristic shape of a mug (e.g., handle, rim for drinking) and does not match the description of a 'sloth-faced wooden mug'.
Step 3
Decision: Unsure
/root/autodl-tmp/pv_dataset/pin_capture/val/00876-mv2HUxq3B53/35/rgb/rgb_s10_far.png
Coarse Category
mug (Cache)
Navigation
front-left (MLLM-Reasoning)
NBV Reasoning (LLM)
Prompt:
You are an embodied perception planner for attribute-based object verification.
**Task**: The following 3 sentences ALL describe the SAME target object. We have decomposed these descriptions into specific attributes to verify.
**Object Description (3 sentences about the SAME object)**:
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
**Object Category**: mug
**Attribute Verification Status**:
[VERIFIED] material: 'wooden'
[CONTRADICTED] color: expected 'white'
[UNVERIFIED] logo_text: 'i'll do it tomorrow'
[CONTRADICTED] relation_1: expected 'a sloth hanged on the rim'
[UNVERIFIED] relation_2: 'a pink heart between the arms'
[VERIFIED] relation_3: 'a leaf near the handle'
[CONTRADICTED] part_attribute: expected 'a sloth - faced wooden mug'
**Navigation State**:
- Current View: You are currently viewing the object from the FRONT
- Already Visited: back-right, front-right, front
- **Available Directions to Move**: ['front-left', 'back', 'back-left']
**Direction Guide** (all directions are relative to YOUR current position):
- front-left: move to YOUR left side (you will see the object from a left angle)
- back: move to the opposite side of the object
- back-left: move to the left side of the opposite view
**IMPORTANT**: You MUST choose ONE direction STRICTLY from the **Available Directions to Move** list above.
- Do NOT choose a direction that is NOT in the list.
- Do NOT choose 'front'; it means "stay in place" and is never a valid navigation direction.
- Do NOT choose a direction that is listed as "Already Visited".
**Your Task**: Analyze each available direction, then choose the best one.
**STEP 1 — Per-Direction Analysis** (you MUST analyze ALL available directions):
For EACH direction in ['front-left', 'back', 'back-left'], assess:
a) **Reachability**: Look at the scene around the object. Is the path in that direction blocked by walls, furniture, or other obstacles? Is there enough space to move there?
b) **Target Visibility**: If you move there, will you be able to see the target object, or might it be occluded by obstacles or the object's own geometry?
c) **Information Gain**: Which missing/unverified attributes could you potentially observe from that angle?
**STEP 2 — Direction Decision**:
Based on your Step 1 analysis, choose the direction that:
1. Is most likely reachable (clear path, no obstacles)
2. Will likely give you a clear view of the target object
3. Has the highest potential to reveal missing/unverified attributes
Output EXACTLY this JSON format:
{"per_direction_analysis": "<your analysis of each available direction>", "chosen_direction": "<must be from Available Directions>", "why": "<brief reason linking to missing attributes and reachability>"}
Response:
{
"per_direction_analysis": "front-left: The path is clear, with no furniture blocking the way. From this angle, the mug's front-left side will be visible, potentially revealing the white text and any nearby leaf or sloth figure. back: The path is clear, but the object will be seen from the rear, which may obscure the sloth and heart details. back-left: The path is clear, but the angle may be partially obstructed by the surrounding chairs, reducing visibility of the mug's features.",
"chosen_direction": "front-left",
"why": "front-left offers the best visibility of the white text and potential leaf near the handle, which are unverified attributes, while maintaining clear reachability and minimal obstruction."
}Model Input (Crop/Ref)
Attribute Verification
| Attribute | Expected | Step Result | Tracker |
|---|---|---|---|
| material | wooden | Matched | Matched |
| color | white | Contradictory | Contradictory |
| logo_text | i'll do it tomorrow | Missing | Missing |
| relation_1 | a sloth hanged on the rim | Missing | Contradictory |
| relation_2 | a pink heart between the arms | Missing | Missing |
| relation_3 | a leaf near the handle | Matched | Matched |
| part_attribute | a sloth - faced wooden mug | Contradictory | Contradictory |
Verification Details (7 items)
Target: material (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
material = wooden
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a mug, and its material is clearly wooden, as evidenced by the visible wood grain texture."
}Reason: The object is a mug, and its material is clearly wooden, as evidenced by the visible wood grain texture.
Target: color (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
color = white
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is not a mug; it is a wooden bucket or container with a sloth figure and a green leaf, not a mug with white text or a sloth hanging on the rim as described in the object descriptions."
}Reason: The object in the image is not a mug; it is a wooden bucket or container with a sloth figure and a green leaf, not a mug with white text or a sloth hanging on the rim as described in the object descriptions.
Target: logo_text (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
logo_text = i'll do it tomorrow
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image is too blurry and pixelated to clearly read any text on the mug. While there is a white area that might be text, it is not legible enough to confirm whether it says 'i'll do it tomorrow'."
}Reason: The image is too blurry and pixelated to clearly read any text on the mug. While there is a white area that might be text, it is not legible enough to confirm whether it says 'i'll do it tomorrow'.
Target: relation_1 (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_1 = a sloth hanged on the rim
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image is too blurry and pixelated to clearly identify whether a sloth is hanging on the rim. The green shape near the handle could be a leaf or part of a design, but it is not clear if it is a sloth or any other figure."
}Reason: The image is too blurry and pixelated to clearly identify whether a sloth is hanging on the rim. The green shape near the handle could be a leaf or part of a design, but it is not clear if it is a sloth or any other figure.
Target: relation_2 (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_2 = a pink heart between the arms
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image is too blurry and pixelated to clearly identify any small figure, heart, or text, making it impossible to verify if a pink heart is between the arms of a sloth."
}Reason: The image is too blurry and pixelated to clearly identify any small figure, heart, or text, making it impossible to verify if a pink heart is between the arms of a sloth.
Target: relation_3 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_3 = a leaf near the handle
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. A small green leaf is visible near the handle, matching the attribute 'a leaf near the handle'."
}Reason: The object is a wooden mug, matching the required category. A small green leaf is visible near the handle, matching the attribute 'a leaf near the handle'.
Target: part_attribute (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
part_attribute = a sloth - faced wooden mug
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object in the image is not a mug; it appears to be a wooden bucket or container with a white substance on top and a green leaf-like object, not matching the description of a mug with a sloth face or text."
}Reason: The object in the image is not a mug; it appears to be a wooden bucket or container with a white substance on top and a green leaf-like object, not matching the description of a mug with a sloth face or text.
Step 4
Decision: Unsure
/root/autodl-tmp/pv_dataset/pin_capture/val/00876-mv2HUxq3B53/35/rgb/rgb_s0_far.png
Coarse Category
mug (Cache)
Navigation
front-left (MLLM-Reasoning)
NBV Reasoning (LLM)
Prompt:
You are an embodied perception planner for attribute-based object verification.
**Task**: The following 3 sentences ALL describe the SAME target object. We have decomposed these descriptions into specific attributes to verify.
**Object Description (3 sentences about the SAME object)**:
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
**Object Category**: mug
**Attribute Verification Status**:
[VERIFIED] material: 'wooden'
[CONTRADICTED] color: expected 'white'
[VERIFIED] logo_text: 'i'll do it tomorrow'
[CONTRADICTED] relation_1: expected 'a sloth hanged on the rim'
[VERIFIED] relation_2: 'a pink heart between the arms'
[VERIFIED] relation_3: 'a leaf near the handle'
[CONTRADICTED] part_attribute: expected 'a sloth - faced wooden mug'
**Navigation State**:
- Current View: You are currently viewing the object from the FRONT
- Already Visited: front, back, back-right, front-right
- **Available Directions to Move**: ['front-left', 'back-left']
**Direction Guide** (all directions are relative to YOUR current position):
- front-left: move to YOUR left side (you will see the object from a left angle)
- back-left: move to the left side of the opposite view
**IMPORTANT**: You MUST choose ONE direction STRICTLY from the **Available Directions to Move** list above.
- Do NOT choose a direction that is NOT in the list.
- Do NOT choose 'front'; it means "stay in place" and is never a valid navigation direction.
- Do NOT choose a direction that is listed as "Already Visited".
**Your Task**: Analyze each available direction, then choose the best one.
**STEP 1 — Per-Direction Analysis** (you MUST analyze ALL available directions):
For EACH direction in ['front-left', 'back-left'], assess:
a) **Reachability**: Look at the scene around the object. Is the path in that direction blocked by walls, furniture, or other obstacles? Is there enough space to move there?
b) **Target Visibility**: If you move there, will you be able to see the target object, or might it be occluded by obstacles or the object's own geometry?
c) **Information Gain**: Which missing/unverified attributes could you potentially observe from that angle?
**STEP 2 — Direction Decision**:
Based on your Step 1 analysis, choose the direction that:
1. Is most likely reachable (clear path, no obstacles)
2. Will likely give you a clear view of the target object
3. Has the highest potential to reveal missing/unverified attributes
Output EXACTLY this JSON format:
{"per_direction_analysis": "<your analysis of each available direction>", "chosen_direction": "<must be from Available Directions>", "why": "<brief reason linking to missing attributes and reachability>"}
Response:
{
"per_direction_analysis": "front-left: The path to the left is clear, with no visible obstructions. From this angle, the mug's left side will be visible, potentially revealing the sloth face, the heart, and the white text more clearly. This direction may help verify the 'sloth-faced' attribute and the spatial relationship of the heart and text. back-left: This direction is less accessible as it leads toward the kitchen area, which is partially occluded by furniture and the refrigerator. The view may be obstructed, and the mug might not be fully visible. Information gain is lower due to occlusion and limited visibility.",
"chosen_direction": "front-left",
"why": "The front-left direction offers clear reachability and visibility, allowing verification of the 'sloth-faced' attribute and spatial relationships (heart and text) that are currently unverified, while avoiding occlusion and obstructions."
}Model Input (Crop/Ref)
Attribute Verification
| Attribute | Expected | Step Result | Tracker |
|---|---|---|---|
| material | wooden | Matched | Matched |
| color | white | Matched | Contradictory |
| logo_text | i'll do it tomorrow | Matched | Matched |
| relation_1 | a sloth hanged on the rim | Matched | Contradictory |
| relation_2 | a pink heart between the arms | Matched | Matched |
| relation_3 | a leaf near the handle | Matched | Matched |
| part_attribute | a sloth - faced wooden mug | Matched | Contradictory |
Verification Details (7 items)
Target: material (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
material = wooden
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a mug, and its material is clearly wooden, as seen in the image."
}Reason: The object is a mug, and its material is clearly wooden, as seen in the image.
Target: color (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
color = white
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. The white text on the mug is visible, satisfying the attribute 'color = white'."
}Reason: The object is a wooden mug, matching the required category. The white text on the mug is visible, satisfying the attribute 'color = white'.
Target: logo_text (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
logo_text = i'll do it tomorrow
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. The white text on the mug reads 'i'll do it tomorrow', which matches the specified attribute."
}Reason: The object is a wooden mug, matching the required category. The white text on the mug reads 'i'll do it tomorrow', which matches the specified attribute.
Target: relation_1 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_1 = a sloth hanged on the rim
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. The image shows a sloth figure hanging on the rim, consistent with the attribute 'a sloth hanged on the rim'."
}Reason: The object is a wooden mug, matching the required category. The image shows a sloth figure hanging on the rim, consistent with the attribute 'a sloth hanged on the rim'.
Target: relation_2 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_2 = a pink heart between the arms
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. The image shows a pink heart positioned between the sloth's arms, which matches the specified attribute 'relation_2 = a pink heart between the arms'."
}Reason: The object is a wooden mug, matching the required category. The image shows a pink heart positioned between the sloth's arms, which matches the specified attribute 'relation_2 = a pink heart between the arms'.
Target: relation_3 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_3 = a leaf near the handle
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. A green leaf is visible near the handle, which matches the attribute 'a leaf near the handle'."
}Reason: The object is a wooden mug, matching the required category. A green leaf is visible near the handle, which matches the attribute 'a leaf near the handle'.
Target: part_attribute (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
part_attribute = a sloth - faced wooden mug
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth face, a pink heart, and white text, matching the attribute 'a sloth-faced wooden mug'."
}Reason: The object is a wooden mug with a sloth face, a pink heart, and white text, matching the attribute 'a sloth-faced wooden mug'.
Step 5
Decision: Unsure
/root/autodl-tmp/pv_dataset/pin_capture/val/00876-mv2HUxq3B53/35/rgb/rgb_s2_far.png
Coarse Category
mug (Cache)
Navigation
front-left (MLLM-Reasoning)
NBV Reasoning (LLM)
Prompt:
You are an embodied perception planner for attribute-based object verification.
**Task**: The following 3 sentences ALL describe the SAME target object. We have decomposed these descriptions into specific attributes to verify.
**Object Description (3 sentences about the SAME object)**:
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
**Object Category**: mug
**Attribute Verification Status**:
[VERIFIED] material: 'wooden'
[CONTRADICTED] color: expected 'white'
[VERIFIED] logo_text: 'i'll do it tomorrow'
[VERIFIED] relation_1: 'a sloth hanged on the rim'
[VERIFIED] relation_2: 'a pink heart between the arms'
[VERIFIED] relation_3: 'a leaf near the handle'
[CONTRADICTED] part_attribute: expected 'a sloth - faced wooden mug'
**Navigation State**:
- Current View: You are currently viewing the object from the FRONT
- Already Visited: front-right, front, back-left, back, back-right
- **Available Directions to Move**: ['front-left']
**Direction Guide** (all directions are relative to YOUR current position):
- front-left: move to YOUR left side (you will see the object from a left angle)
**IMPORTANT**: You MUST choose ONE direction STRICTLY from the **Available Directions to Move** list above.
- Do NOT choose a direction that is NOT in the list.
- Do NOT choose 'front'; it means "stay in place" and is never a valid navigation direction.
- Do NOT choose a direction that is listed as "Already Visited".
**Your Task**: Analyze each available direction, then choose the best one.
**STEP 1 — Per-Direction Analysis** (you MUST analyze ALL available directions):
For EACH direction in ['front-left'], assess:
a) **Reachability**: Look at the scene around the object. Is the path in that direction blocked by walls, furniture, or other obstacles? Is there enough space to move there?
b) **Target Visibility**: If you move there, will you be able to see the target object, or might it be occluded by obstacles or the object's own geometry?
c) **Information Gain**: Which missing/unverified attributes could you potentially observe from that angle?
**STEP 2 — Direction Decision**:
Based on your Step 1 analysis, choose the direction that:
1. Is most likely reachable (clear path, no obstacles)
2. Will likely give you a clear view of the target object
3. Has the highest potential to reveal missing/unverified attributes
Output EXACTLY this JSON format:
{"per_direction_analysis": "<your analysis of each available direction>", "chosen_direction": "<must be from Available Directions>", "why": "<brief reason linking to missing attributes and reachability>"}
Response:
{
"per_direction_analysis": "front-left: The path to the front-left is clear, with no visible obstacles blocking the way. Moving to the front-left will provide a left-side angle of the mug, which may reveal the white text and sloth figure more clearly from a different perspective. This angle could help verify the 'sloth - faced wooden mug' attribute and potentially confirm the position of the pink heart and leaf relative to the rim and handle, which are currently inferred from the front view.",
"chosen_direction": "front-left",
"why": "The front-left direction offers a clear path and a new viewing angle that can help verify the 'sloth - faced wooden mug' attribute and potentially confirm the spatial relationship of the pink heart and leaf, which are critical for attribute verification and are not fully confirmed from the current front view."
}Model Input (Crop/Ref)
Attribute Verification
| Attribute | Expected | Step Result | Tracker |
|---|---|---|---|
| material | wooden | Matched | Matched |
| color | white | Contradictory | Contradictory |
| logo_text | i'll do it tomorrow | Matched | Matched |
| relation_1 | a sloth hanged on the rim | Matched | Matched |
| relation_2 | a pink heart between the arms | Matched | Matched |
| relation_3 | a leaf near the handle | Missing | Matched |
| part_attribute | a sloth - faced wooden mug | Matched | Contradictory |
Verification Details (7 items)
Target: material (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
material = wooden
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a mug, and its material is clearly wooden, as indicated by the visible wood grain and texture in the image."
}Reason: The object is a mug, and its material is clearly wooden, as indicated by the visible wood grain and texture in the image.
Target: color (Contradictory)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
color = white
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "No",
"reason": "The object is not a mug; it is a wooden container with a sloth figure and heart design, which does not match the category 'mug'."
}Reason: The object is not a mug; it is a wooden container with a sloth figure and heart design, which does not match the category 'mug'.
Target: logo_text (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
logo_text = i'll do it tomorrow
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth figure and pink heart, and the white text 'i'll do it tomorrow' is visible on the mug, matching the required attribute."
}Reason: The object is a wooden mug with a sloth figure and pink heart, and the white text 'i'll do it tomorrow' is visible on the mug, matching the required attribute.
Target: relation_1 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_1 = a sloth hanged on the rim
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth figure hanging on the rim, matching the attribute 'a sloth hanged on the rim'."
}Reason: The object is a wooden mug with a sloth figure hanging on the rim, matching the attribute 'a sloth hanged on the rim'.
Target: relation_2 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_2 = a pink heart between the arms
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth figure on it, and a pink heart is clearly visible positioned between the sloth's arms."
}Reason: The object is a wooden mug with a sloth figure on it, and a pink heart is clearly visible positioned between the sloth's arms.
Target: relation_3 (Missing)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_3 = a leaf near the handle
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Unsure",
"reason": "The image is too blurry and pixelated to clearly identify a leaf near the handle. While there is a small figure (sloth) and a pink heart, the presence and location of a leaf relative to the handle cannot be confirmed."
}Reason: The image is too blurry and pixelated to clearly identify a leaf near the handle. While there is a small figure (sloth) and a pink heart, the presence and location of a leaf relative to the handle cannot be confirmed.
Target: part_attribute (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
part_attribute = a sloth - faced wooden mug
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth face, a pink heart, and white text, matching the attribute 'a sloth-faced wooden mug'."
}Reason: The object is a wooden mug with a sloth face, a pink heart, and white text, matching the attribute 'a sloth-faced wooden mug'.
Step 6
Decision: Yes
/root/autodl-tmp/pv_dataset/pin_capture/val/00876-mv2HUxq3B53/35/rgb/rgb_s4_far.png
Coarse Category
mug (Cache)
Navigation
front
Model Input (Crop/Ref)
Attribute Verification
| Attribute | Expected | Step Result | Tracker |
|---|---|---|---|
| material | wooden | Matched | Matched |
| color | white | Matched | Contradictory |
| logo_text | i'll do it tomorrow | Matched | Matched |
| relation_1 | a sloth hanged on the rim | Matched | Matched |
| relation_2 | a pink heart between the arms | Matched | Matched |
| relation_3 | a leaf near the handle | Matched | Matched |
| part_attribute | a sloth - faced wooden mug | Matched | Matched |
Verification Details (7 items)
Target: material (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
material = wooden
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a mug, and its material is clearly wooden, as evidenced by the visible wood grain texture and the overall shape of the mug."
}Reason: The object is a mug, and its material is clearly wooden, as evidenced by the visible wood grain texture and the overall shape of the mug.
Target: color (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
color = white
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. The white text on the mug is visible, fulfilling the 'color = white' attribute requirement."
}Reason: The object is a wooden mug, matching the required category. The white text on the mug is visible, fulfilling the 'color = white' attribute requirement.
Target: logo_text (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
logo_text = i'll do it tomorrow
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth figure and white text that reads 'i'll do it tomorrow', matching the required attribute."
}Reason: The object is a wooden mug with a sloth figure and white text that reads 'i'll do it tomorrow', matching the required attribute.
Target: relation_1 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_1 = a sloth hanged on the rim
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. The image shows a sloth figure hanging on the rim, consistent with the attribute 'a sloth hanged on the rim'."
}Reason: The object is a wooden mug, matching the required category. The image shows a sloth figure hanging on the rim, consistent with the attribute 'a sloth hanged on the rim'.
Target: relation_2 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_2 = a pink heart between the arms
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth figure on the rim, and a pink heart is visible between the sloth's arms, matching the attribute description."
}Reason: The object is a wooden mug with a sloth figure on the rim, and a pink heart is visible between the sloth's arms, matching the attribute description.
Target: relation_3 (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
relation_3 = a leaf near the handle
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug, matching the required category. A small green leaf is visible near the handle, matching the attribute 'a leaf near the handle'."
}Reason: The object is a wooden mug, matching the required category. A small green leaf is visible near the handle, matching the attribute 'a leaf near the handle'.
Target: part_attribute (Matched)
Q:
You will be given an image of a candidate object.
We need to verify BOTH the object category AND a specific attribute.
**IMPORTANT**: The object must be the correct CATEGORY first. If it's not a mug, answer 'No' immediately.
Object Category (MUST match): mug
Object descriptions (3 sentences describing the SAME object):
1) a wooden muug with a white text and a small figure on it
2) a wooden mug having a sloth hanged on the rim with a pink heart between the arms, the white writing i'll do it tomorrow and a leaf near the handle
3) a sloth - faced wooden mug with a heart and a white writing on it
Attribute to verify:
part_attribute = a sloth - faced wooden mug
Answer in JSON format:
{"answer": "Yes/No/Unsure", "reason": "brief explanation"}
Rules:
- FIRST check: Is the object in the image a mug? If NOT, say 'No'.
- If it IS the correct category, then check the attribute.
- If the attribute matches the expected value, say 'Yes'.
- If the attribute is visible but has a DIFFERENT value (e.g. red instead of blue), say 'No'.
- If the attribute is NOT visible due to angle/occlusion, say 'Unsure'.
A:
{
"answer": "Yes",
"reason": "The object is a wooden mug with a sloth-faced design, a pink heart, and white text, matching the attribute 'a sloth-faced wooden mug'."
}Reason: The object is a wooden mug with a sloth-faced design, a pink heart, and white text, matching the attribute 'a sloth-faced wooden mug'.