probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT > Class Template Reference

#include "probfd/algorithms/heuristic_search_base.h"

Inheritance diagram for probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >:

Description

template<typename State, typename Action, typename StateInfoT>
class probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >

The common base class for MDP h search algorithms.

Template Parameters

State	- The state type of the underlying MDP model.
Action	- The action type of the undelying MDP model.
StateInfoT	- The state information container type.

Public Member Functions
Interval	lookup_bounds (StateID state_id) const
	Looks up the current value interval of state_id.
bool	was_visited (StateID state_id) const
	Checks if the state represented by state_id has been visited yet.
AlgorithmValueType	compute_bellman (CostFunctionType &cost_function, StateID state_id, const std::vector< TransitionType > &transitions, value_t termination_cost) const
	Computes the Bellman operator value for a state.
AlgorithmValueType	compute_bellman_and_greedy (CostFunctionType &cost_function, StateID state_id, std::vector< TransitionType > &transitions, value_t termination_cost, std::vector< AlgorithmValueType > &qvalues, value_t epsilon=g_epsilon) const
	Computes the Bellman operator value for a state, as well as all transitions achieving a value epsilon-close to the minimum value and their computed Q-values.
std::optional< TransitionType >	select_greedy_transition (MDPType &mdp, std::optional< Action > previous_greedy_action, std::vector< TransitionType > &greedy_transitions)
	Selects a greedy transition from the given list of greedy transitions through the policy selector passed on construction.
bool	update_value (StateInfo &state_info, AlgorithmValueType other, value_t epsilon=g_epsilon)
	Updates the value of the state associated with the given storage.
bool	update_policy (StateInfo &state_info, const std::optional< TransitionType > &transition)
	Updates the current greedy action of the state associated with the given storage.

Member Function Documentation

lookup_bounds()

template<typename State , typename Action , typename StateInfoT >

Interval probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >::lookup_bounds ( StateID state_id ) const

nodiscard

Looks up the current value interval of state_id.

was_visited()

template<typename State , typename Action , typename StateInfoT >

bool probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >::was_visited ( StateID state_id ) const

nodiscard

Checks if the state represented by state_id has been visited yet.

compute_bellman()

template<typename State , typename Action , typename StateInfoT >

auto probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >::compute_bellman	(	CostFunctionType &	cost_function,
		StateID	state_id,
		const std::vector< TransitionType > &	transitions,
		value_t	termination_cost ) const

Computes the Bellman operator value for a state.

compute_bellman_and_greedy()

template<typename State , typename Action , typename StateInfoT >

auto probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >::compute_bellman_and_greedy	(	CostFunctionType &	cost_function,
		StateID	state_id,
		std::vector< TransitionType > &	transitions,
		value_t	termination_cost,
		std::vector< AlgorithmValueType > &	qvalues,
		value_t	epsilon = g_epsilon ) const

Computes the Bellman operator value for a state, as well as all transitions achieving a value epsilon-close to the minimum value and their computed Q-values.

Parameters

[in,out]	transitions	The set of transition to compute the Bellman operator for. The greedy transitions are returned through this parameter by erasing all non-greedy transitions. All greedy transitions will maintain their relative order. If no transition achieves a value lower than the termination cost, the empty list is returned, regardless of the epsilon parameter.
[out]	qvalues	The Q-values are added to this list, which must be empty prior to the call, in the order that matches the greedy transitions returned in transitions .

select_greedy_transition()

template<typename State , typename Action , typename StateInfoT >

auto probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >::select_greedy_transition	(	MDPType &	mdp,
		std::optional< Action >	previous_greedy_action,
		std::vector< TransitionType > &	greedy_transitions )

Selects a greedy transition from the given list of greedy transitions through the policy selector passed on construction.

If the transition list is empty, returns std::nullopt, otherwise some transition from the list.

Attention

The selected transition is moved from the transition list to avoid a copy.

update_value()

template<typename State , typename Action , typename StateInfoT >

bool probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >::update_value	(	StateInfo &	state_info,
		AlgorithmValueType	other,
		value_t	epsilon = g_epsilon )

Updates the value of the state associated with the given storage.

Returns

True if the value has changed by more than the given epsilon, otherwise false.

update_policy()

template<typename State , typename Action , typename StateInfoT >
requires (StorePolicy)

bool probfd::algorithms::heuristic_search::HeuristicSearchBase< State, Action, StateInfoT >::update_policy	(	StateInfo &	state_info,
		const std::optional< TransitionType > &	transition )

Updates the current greedy action of the state associated with the given storage.

Returns true if the greedy action has changed and false otherwise.