GetMasks2DFromExemplar Struct
Segment an image using SAM3 with multimodal prompts. More...
Declaration
Included Headers
Base struct
| class | AsyncBehaviorBase |
|
A base class for behaviors which need to asynchronously run a function that might take a long time to complete. More... | |
Public Constructors Index
| GetMasks2DFromExemplar (const std::string &name, const BT::NodeConfiguration &config, const std::shared_ptr< moveit_pro::behaviors::BehaviorContext > &shared_resources, std::unique_ptr< InferenceHandle > inference_handle=std::make_unique< SAM3DetectHandle >()) | |
|
Constructor for SAM3 multimodal segmentation behavior. More... | |
Public Destructor Index
| ~GetMasks2DFromExemplar () override | |
Private Member Functions Index
| tl::expected< bool, std::string > | doWork () override |
|
Execute multimodal segmentation. More... | |
| tl::expected< void, std::string > | doHalt () override |
|
Cancel an in-flight SAM3 inference via the inference handle. More... | |
| std::shared_future< tl::expected< bool, std::string > > & | getFuture () override |
|
Classes derived from AsyncBehaviorBase must implement getFuture() More... | |
Private Member Attributes Index
| std::unique_ptr< InferenceHandle > | inference_handle_ |
| std::optional< internal::Sam3ModelPorts > | latched_ports_ |
| bool | cpu_fallback_warning_published_ { false } |
| std::shared_future< tl::expected< bool, std::string > > | future_ |
|
Classes derived from AsyncBehaviorBase must have this shared_future as a class member. More... | |
Public Static Functions Index
| static BT::PortsList | providedPorts () |
|
Provides port definitions for BehaviorTree. More... | |
| static BT::KeyValueVector | metadata () |
|
Provides behavior metadata for UI. More... | |
Description
Segment an image using SAM3 with multimodal prompts.
This behavior segments an image using text, an optional exemplar image, and/or bounding box prompts. Supports any combination of prompt types for flexible segmentation.
| Data Port Name | Port Type | Object Type |
|---|---|---|
| target_image | input | sensor_msgs::msg::Image |
| text_prompt | input | std::string |
| exemplar_image | input | sensor_msgs::msg::Image (optional) |
| target_bboxes | input | std::vector<vision_msgs::msg::BoundingBox2D> |
| exemplar_bboxes | input | std::vector<vision_msgs::msg::BoundingBox2D> |
| model_bundle_manifest | input | std::string |
| runtime_id | input | std::string |
| model_package | input | std::string |
| confidence_threshold | input | double (optional, default 0.0) |
| masks2d | output | std::vector<moveit_studio_vision_msgs::msg::Mask2D> |
| mask_count | output | int |
| confidence_scores | output | std::vector<double> |
| confidence_scores_str | output | std::vector<std::string> |
Definition at line 181 of file get_masks_2d_from_exemplar.hpp.
Public Constructors
GetMasks2DFromExemplar()
|
Constructor for SAM3 multimodal segmentation behavior.
- Parameters
-
name The name of a particular instance of this Behavior. This will be set by the behavior tree factory when this Behavior is created within a new behavior tree.
config This contains runtime configuration info for this Behavior, such as the mapping between the Behavior's data ports on the behavior tree's blackboard and the Behavior's input/output ports.
shared_resources Provides access to common resources such as the node handle, failure logger, and TF buffer that are shared between all behaviors inheriting from moveit_pro::behaviors::SharedResourcesNode.
inference_handle Custom inference implementation (default uses SAM3DetectHandle)
Declaration at line 290 of file get_masks_2d_from_exemplar.hpp, definition at line 605 of file get_masks_2d_from_exemplar.cpp.
Public Destructor
~GetMasks2DFromExemplar()
|
Definition at line 294 of file get_masks_2d_from_exemplar.hpp.
Private Member Functions
doHalt()
| virtual |
Cancel an in-flight SAM3 inference via the inference handle.
Forwards to InferenceHandle::requestTerminate(). The selected runtime observes cancellation before each graph invocation and may also terminate an in-flight graph. doWork() creates a fresh request control before invoking predict().
Declaration at line 321 of file get_masks_2d_from_exemplar.hpp, definition at line 867 of file get_masks_2d_from_exemplar.cpp.
doWork()
| virtual |
Execute multimodal segmentation.
- Returns
Success or error message
Declaration at line 313 of file get_masks_2d_from_exemplar.hpp, definition at line 669 of file get_masks_2d_from_exemplar.cpp.
getFuture()
| inline virtual |
Classes derived from AsyncBehaviorBase must implement getFuture()
Definition at line 335 of file get_masks_2d_from_exemplar.hpp.
Private Member Attributes
cpu_fallback_warning_published_
|
True once the CPU-inference warning has been published, so it fires once per behavior instance rather than on every tick (load() is called from every doWork()). A bool suffices because the model latches at first load, so the served device cannot change afterwards.
Definition at line 332 of file get_masks_2d_from_exemplar.hpp.
future_
|
Classes derived from AsyncBehaviorBase must have this shared_future as a class member.
Definition at line 341 of file get_masks_2d_from_exemplar.hpp.
inference_handle_
|
Definition at line 322 of file get_masks_2d_from_exemplar.hpp.
latched_ports_
|
The port values the model was loaded from, set on the first successful load. The model latches: a later tick naming different values fails the Behavior rather than swapping models mid-run, so this is what that comparison is made against.
Definition at line 327 of file get_masks_2d_from_exemplar.hpp.
Public Static Functions
metadata()
| static |
Provides behavior metadata for UI.
- Returns
Metadata key-value pairs
Declaration at line 306 of file get_masks_2d_from_exemplar.hpp, definition at line 663 of file get_masks_2d_from_exemplar.cpp.
providedPorts()
| static |
Provides port definitions for BehaviorTree.
- Returns
List of input and output ports
Declaration at line 300 of file get_masks_2d_from_exemplar.hpp, definition at line 616 of file get_masks_2d_from_exemplar.cpp.
The documentation for this struct was generated from the following files:
Generated via doxygen2docusaurus 2.2.2 by Doxygen 1.9.8.