moveit_studio::behaviors::GetMasks2DFromTextQuery Class Reference

Segment an image using the ClipSeg model. More...

#include <clipseg_segmentation.hpp>

Inheritance diagram for moveit_studio::behaviors::GetMasks2DFromTextQuery:

Collaboration diagram for moveit_studio::behaviors::GetMasks2DFromTextQuery:

Public Member Functions
	GetMasks2DFromTextQuery (const std::string &name, const BT::NodeConfiguration &config, const std::shared_ptr< moveit_studio::behaviors::BehaviorContext > &shared_resources, std::unique_ptr< GetMasks2DFromTextQueryInterface > clipseg_interface=std::make_unique< GetMasks2DFromTextQueryInterface >())
	Constructor for the clipseg_segmentation behavior.
Public Member Functions inherited from moveit_studio::behaviors::AsyncBehaviorBase
	AsyncBehaviorBase (const std::string &name, const BT::NodeConfiguration &config, const std::shared_ptr< BehaviorContext > &shared_resources)
virtual	~AsyncBehaviorBase ()=default
BT::NodeStatus	onStart () override
	Required implementation of BT::StatefulActionNode::onStart().
BT::NodeStatus	onRunning () override
	Required implementation of BT::StatefulActionNode::onRunning().
void	onHalted () override
	Required implementation of BT::StatefulActionNode::onHalted().
void	resetStatus ()
	Resets the internal status of this node.
Public Member Functions inherited from moveit_studio::behaviors::SharedResourcesNode< BT::StatefulActionNode >
	SharedResourcesNode (const std::string &name, const BT::NodeConfiguration &config, const std::shared_ptr< BehaviorContext > &shared_resources)
	Constructor for SharedResourcesNode. Called by BT::BehaviorTreeFactory when creating a new behavior tree containing this node.

Static Public Member Functions
static BT::PortsList	providedPorts ()
	Implementation of the required providedPorts() function for the sam2_segmentation Behavior.
static BT::KeyValueVector	metadata ()
	Implementation of the metadata() function for displaying metadata, such as Behavior description and subcategory, in the MoveIt Studio Developer Tool.

Protected Member Functions
tl::expected< bool, std::string >	doWork () override
	User-implemented function which handles executing the potentially-long-running process.
void	calculate_logits (const std::vector< std::string > &prompts, const std::vector< std::string > &negative_prompts, moveit_pro_ml::ONNXImage &logits) const
std::shared_future< tl::expected< bool, std::string > > &	getFuture () override
	Classes derived from AsyncBehaviorBase must implement getFuture() so that it returns a shared_future class member.
Protected Member Functions inherited from moveit_studio::behaviors::AsyncBehaviorBase
virtual tl::expected< void, std::string >	doHalt ()
	Optionally implement additional work needed to cleanly interrupt the async process.
void	notifyCanHalt ()
	Called when runAsync() finishes to notify onHalted() that the async process has finished.

Protected Attributes
std::shared_ptr< GetMasks2DFromTextQueryInterface >	clipseg_interface_
moveit_pro_ml::ONNXImage	onnx_image_
sensor_msgs::msg::Image	image_
std::shared_future< tl::expected< bool, std::string > >	future_
	Classes derived from AsyncBehaviorBase must have this shared_future as a class member.
Protected Attributes inherited from moveit_studio::behaviors::SharedResourcesNode< BT::StatefulActionNode >
std::shared_ptr< BehaviorContext >	shared_resources_

Detailed Description

Segment an image using the ClipSeg model.

Constructor & Destructor Documentation

GetMasks2DFromTextQuery()

moveit_studio::behaviors::GetMasks2DFromTextQuery::GetMasks2DFromTextQuery	(	const std::string &	name,
		const BT::NodeConfiguration &	config,
		const std::shared_ptr< moveit_studio::behaviors::BehaviorContext > &	shared_resources,
		std::unique_ptr< GetMasks2DFromTextQueryInterface >	clipseg_interface = std::make_unique<GetMasks2DFromTextQueryInterface>()
	)

Constructor for the clipseg_segmentation behavior.

Parameters

name	The name of a particular instance of this Behavior. This will be set by the behavior tree factory when this Behavior is created within a new behavior tree.
config	This contains runtime configuration info for this Behavior, such as the mapping between the Behavior's data ports on the behavior tree's blackboard. This will be set by the behavior tree factory when this Behavior is created within a new behavior tree.

An important limitation is that the members of the base Behavior class are not instantiated until after the initialize() function is called, so these classes should not be used within the constructor.

Member Function Documentation

calculate_logits()

void moveit_studio::behaviors::GetMasks2DFromTextQuery::calculate_logits ( const std::vector< std::string > &  prompts, const std::vector< std::string > &  negative_prompts, moveit_pro_ml::ONNXImage &  logits  ) const

protected

doWork()

tl::expected< bool, std::string > moveit_studio::behaviors::GetMasks2DFromTextQuery::doWork ( )

overrideprotectedvirtual

User-implemented function which handles executing the potentially-long-running process.

This function is called within an async process in a separate thread.

Returns

A tl::expected which contains a bool indicating task success if the process completed successfully or was canceled, or an error message if the process failed unexpectedly.

Implements moveit_studio::behaviors::AsyncBehaviorBase.

getFuture()

std::shared_future< tl::expected< bool, std::string > > & moveit_studio::behaviors::GetMasks2DFromTextQuery::getFuture ( )

inlineoverrideprotectedvirtual

Classes derived from AsyncBehaviorBase must implement getFuture() so that it returns a shared_future class member.

Implements moveit_studio::behaviors::AsyncBehaviorBase.

metadata()

BT::KeyValueVector moveit_studio::behaviors::GetMasks2DFromTextQuery::metadata ( )

static

Implementation of the metadata() function for displaying metadata, such as Behavior description and subcategory, in the MoveIt Studio Developer Tool.

Returns

A BT::KeyValueVector containing the Behavior metadata.

providedPorts()

BT::PortsList moveit_studio::behaviors::GetMasks2DFromTextQuery::providedPorts ( )

static

Implementation of the required providedPorts() function for the sam2_segmentation Behavior.

The BehaviorTree.CPP library requires that Behaviors must implement a static function named providedPorts() which defines their input and output ports. If the Behavior does not use any ports, this function must return an empty BT::PortsList. This function returns a list of ports with their names and port info, which is used internally by the behavior tree.

Returns

sam2_segmentation does not use expose any ports, so this function returns an empty list.

Member Data Documentation

clipseg_interface_

std::shared_ptr<GetMasks2DFromTextQueryInterface> moveit_studio::behaviors::GetMasks2DFromTextQuery::clipseg_interface_

protected

future_

std::shared_future<tl::expected<bool, std::string> > moveit_studio::behaviors::GetMasks2DFromTextQuery::future_

protected

Classes derived from AsyncBehaviorBase must have this shared_future as a class member.

image_

sensor_msgs::msg::Image moveit_studio::behaviors::GetMasks2DFromTextQuery::image_

protected

onnx_image_

moveit_pro_ml::ONNXImage moveit_studio::behaviors::GetMasks2DFromTextQuery::onnx_image_

protected

---

The documentation for this class was generated from the following files:

• clipseg_segmentation.hpp
• clipseg_segmentation.cpp