probfd::algorithms::PolicyPicker< State, Action > Class Template Reference

#include "probfd/algorithms/policy_picker.h"

Inheritance diagram for probfd::algorithms::PolicyPicker< State, Action >:

Description

template<typename State, typename Action>
class probfd::algorithms::PolicyPicker< State, Action >

An strategy interface used to choose break ties between multiple greedy actions for a state.

This interface is used by heuristic search algorithms which maintain a greedy policy during search. If there are multiple greedy actions in a state after a value update has been performed and the policy needs to be adjusted, an implementation of this interface breaks the ties by choosing one of the candidate greedy actions.

Example

class AlwaysFirstPicker : public PolicyPicker<OperatorID> {
protected:
    int pick_index(
        MDP<State, OperatorID>&,
        StateID,
        OperatorID,
        const std::vector<OperatorID>&,
        const std::vector<Distribution<StateID>>&,
        StateProperties&) override
    {
        return 0; // Always choose the first greedy action in the list
    }
};

Template Parameters

Action

- The action type of the underlying MDP model.

Public Member Functions
virtual int	pick_index (MDP< State, Action > &mdp, std::optional< Action > incumbent_greedy, const std::vector< Transition< Action > > &greedy_transitions, StateProperties &properties)=0
	Selects a greedy transition from multiple candidates by returning its index in the candidate list.
virtual void	print_statistics (std::ostream &)
	Prints statistics to an output stream, e.g. the number of queries made to the interface.

Member Function Documentation

pick_index()

template<typename State , typename Action >

virtual int probfd::algorithms::PolicyPicker< State, Action >::pick_index ( MDP< State, Action > & mdp, std::optional< Action > incumbent_greedy, const std::vector< Transition< Action > > & greedy_transitions, StateProperties & properties )

pure virtual

Selects a greedy transition from multiple candidates by returning its index in the candidate list.

Parameters

mdp	- The MDP in which the search is performed.
incumbent_greedy	- The incumbent greedy action.
greedy_transitions	- A list of candidate greedy transitions.
properties	- The interface to the heuristic search algorithm. Can be used to query additional information about the involved states and actions.

Implemented in probfd::policy_pickers::StablePolicyPicker< State, Action, Derived >, probfd::policy_pickers::StablePolicyPicker< quotients::QuotientState< int, const ProbabilisticTransition * >, quotients::QuotientAction< const ProbabilisticTransition * >, ArbitraryTiebreaker< quotients::QuotientState< int, const ProbabilisticTransition * >, quotients::QuotientAction< const ProbabilisticTransition * > > >, probfd::policy_pickers::StablePolicyPicker< State, Action, ArbitraryTiebreaker< State, Action > >, probfd::policy_pickers::StablePolicyPicker< State, Action, RandomTiebreaker< State, Action > >, probfd::policy_pickers::StablePolicyPicker< State, Action, VDiffTiebreaker< State, Action > >, and probfd::policy_pickers::StablePolicyPicker< State, OperatorID, OperatorIdTiebreaker >.

print_statistics()

template<typename State , typename Action >

virtual void probfd::algorithms::PolicyPicker< State, Action >::print_statistics ( std::ostream & )

inlinevirtual

Prints statistics to an output stream, e.g. the number of queries made to the interface.