Classes
struct	NonTaskWorkerParameters

Public Member Functions
Results	RunMajorityVote (IList< Datum > data, IList< Datum > fullData, bool calculateAccuracy, bool useVoteDistribution)
	Runs the majority vote method on the data. More...

Results	RunDawidSkene (IList< Datum > data, IList< Datum > fullData, bool calculateAccuracy)
	Run Dawid-Skene on the data. More...

void	RunBCC (string modelName, IList< Datum > data, IList< Datum > fullData, BCC model, RunMode mode, bool calculateAccuracy, int numCommunities=-1, bool serialize=false, bool serializeCommunityPosteriors=false)

void	WriteBasicStatistics (StreamWriter writer)

virtual void	WriteResults (StreamWriter writer, bool writeCommunityParameters, bool writeWorkerParameters, bool writeWorkerCommunities)

virtual void	WriteAccuracy (StreamWriter writer)

double[,]	GetConfusionMatrices (string workerId="", int communityIndex=-1)

Discrete	getTaskTrueLabel (string taskId)
	Get True Label values of the given taskId More...

Static Public Member Functions
static double[]	GetConfusionMatrixDiagonal (Vector[] confusionMatrixMean)

static void	WriteConfusionMatrix (StreamWriter writer, string worker, Dirichlet[] confusionMatrix)

static void	WriteWorkerConfusionMatrix (StreamWriter writer, string worker, double[,] confusionMatrix)

static void	WriteWorkerConfusionMatrix (StreamWriter writer, string worker, Vector[] confusionMatrix)

static string	GetModelName (string dataset, RunType runType, TaskSelectionMethod taskSelectionMetric, WorkerSelectionMethod workerSelectionMetric, bool online, int taskSamples=-1, int workerSamples=-1, int numCommunities=-1)

Protected Member Functions
virtual void	ClearResults ()

virtual void	UpdateResults (BCCPosteriors posteriors, RunMode mode)

virtual void	UpdateAccuracy ()
	Updates the accuracy using the current results. More...

Properties
Dictionary< string, Discrete >	TrueLabel `[get, protected set]`
	The posterior of the true label for each task. More...

Dictionary< string, Discrete >	LookAheadTrueLabel `[get, protected set]`
	The predicted label for each task when doing simulations from the current model state. It avoids overwriting the true label posterior. More...

Dictionary< string, Discrete >	TrueLabelConstraint `[get, protected set]`
	The posterior for the constraint that allows online learning for the true label variable. More...

Dictionary< string, int?>	PredictedLabel `[get, protected set]`
	The predicted label for each task More...

Dirichlet	BackgroundLabelProb `[get, protected set]`
	The probabilities that generate the true label of all the tasks. More...

Dictionary< string, Dirichlet[]>	WorkerConfusionMatrix `[get, protected set]`
	The posterior of the confusion matrix of each worker. More...

Dictionary< string, Vector[]>	WorkerConfusionMatrixMean `[get, protected set]`
	The mean of the posterior of the confusion matrix of each worker. More...

Dictionary< string, Dirichlet[]>	LookAheadWorkerConfusionMatrix `[get, protected set]`
	The look-ahead posterior of the confusion matrix of each worker obtained after simulating a new label in look-ahead run mode. More...

Dictionary< string, Dictionary< string, Discrete > >	WorkerPrediction `[get, protected set]`
	The predictive probabilities of the labels produced by each worker. More...

Dictionary< string, Discrete >	WorkerCommunity `[get, protected set]`
	The community membership probabilities of each worker. More...

Dirichlet[][]	CommunityConfusionMatrix `[get, protected set]`
	The confusion matrix of each community. More...

VectorGaussian[][]	CommunityScoreMatrix `[get, protected set]`
	The score matrix of each community. More...

Dictionary< string, VectorGaussian[]>	WorkerScoreMatrixConstraint `[get, protected set]`
	The posterior for the constraint that allows online learning for worker confusion matrices int the community model. More...

Dirichlet	CommunityProb `[get, protected set]`
	The probabilities that generate the community memberships of all the workers. More...

Dictionary< string, Discrete >	CommunityConstraint `[get, protected set]`
	The posterior for the constraint that allows online learning for community membership. int the community model. More...

Bernoulli	ModelEvidence `[get, protected set]`
	Model evidence. More...

DataMapping	Mapping `[get, set]`
	The data mapping. More...

DataMapping	FullMapping `[get, set]`
	The full data mapping. More...

Dictionary< string, int?>	GoldLabels `[get, protected set]`
	The gold labels of each task. The gold label type is nullable to support the (usual) situation where the is no labels. More...

double	Accuracy `[get]`
	The accuracy of the current true label predictions. More...

double	WorkerLabelAccuracy `[get, protected set]`
	The accuracy of the worker labels. More...

double	NegativeLogProb `[get]`
	The negative log probability density (NLPD) scores of the current true label predictions. More...

double	AvgRecall `[get]`
	The average recall of the current true label predictions. More...

double[,]	ModelConfusionMatrix `[get]`
	The confusion matrix of the predicted true labels against the gold labels The rows are the gold labels and the columns are the predicted labels. More...

bool	IsCommunityModel `[get]`
	Flags whether the model instance is CBCC (true) or BCC (false). More...

bool	IsTimeModel `[get]`
	Flags whether the model instance is a BCC time model (true) or not (false). More...

bool	IsTimeMultimodeModel `[get]`
	Flags whether the model instance is a BCC time model (true) or not (false). More...

bool	IsTimeTaskPropensityModel `[get]`

int	CommunityCount `[get]`
	The number of communities. More...

ConfusionMatrix	ResultsConfusionMatrixForBinaryLabels `[get]`

ReceiverOperatingCharacteristic	RocCurve `[get]`

Detailed Description

Results class containing posteriors and predictions.

Member Function Documentation

double [,] CrowdsourcingModels.Results.GetConfusionMatrices	(	string	workerId = `""`,
		int	communityIndex = `-1`
	)

inline

Parameters

workerId	if the workerId is null and communityIndex > -1, return the community confusion matrix; otherwise return the worker confusion matrix
communityIndex

Returns

Discrete CrowdsourcingModels.Results.getTaskTrueLabel ( string taskId )

inline

Get True Label values of the given taskId

Parameters

taskId

Returns

Results CrowdsourcingModels.Results.RunDawidSkene	(	IList< Datum >	data,
		IList< Datum >	fullData,
		bool	calculateAccuracy
	)

inline

Run Dawid-Skene on the data.

Parameters

data	The data.
calculateAccuracy	Whether to calculate accuracy

Returns: A results instance

Results CrowdsourcingModels.Results.RunMajorityVote	(	IList< Datum >	data,
		IList< Datum >	fullData,
		bool	calculateAccuracy,
		bool	useVoteDistribution
	)

inline

Runs the majority vote method on the data.

Parameters

data	The data
calculateAccuracy	Compute the accuracy (true).
useVoteDistribution	The true label is sampled from the vote distribution (true) or it is taken as the mode of the vote counts (false). In the latter case, ties are broken by sampling from the most voted classes.

Returns: The updated results

virtual void CrowdsourcingModels.Results.UpdateAccuracy ( )

inlineprotectedvirtual

Updates the accuracy using the current results.

Property Documentation

double CrowdsourcingModels.Results.Accuracy

get

The accuracy of the current true label predictions.

double CrowdsourcingModels.Results.AvgRecall

get

The average recall of the current true label predictions.

Dirichlet CrowdsourcingModels.Results.BackgroundLabelProb

getprotected set

The probabilities that generate the true label of all the tasks.

Dirichlet [][] CrowdsourcingModels.Results.CommunityConfusionMatrix

getprotected set

The confusion matrix of each community.

Dictionary<string, Discrete> CrowdsourcingModels.Results.CommunityConstraint

getprotected set

The posterior for the constraint that allows online learning for community membership. int the community model.

int CrowdsourcingModels.Results.CommunityCount

get

The number of communities.

Dirichlet CrowdsourcingModels.Results.CommunityProb

getprotected set

The probabilities that generate the community memberships of all the workers.

VectorGaussian [][] CrowdsourcingModels.Results.CommunityScoreMatrix

getprotected set

The score matrix of each community.

DataMapping CrowdsourcingModels.Results.FullMapping

getset

The full data mapping.

Dictionary<string, int?> CrowdsourcingModels.Results.GoldLabels

getprotected set

The gold labels of each task. The gold label type is nullable to support the (usual) situation where the is no labels.

bool CrowdsourcingModels.Results.IsCommunityModel

get

Flags whether the model instance is CBCC (true) or BCC (false).

bool CrowdsourcingModels.Results.IsTimeModel

get

Flags whether the model instance is a BCC time model (true) or not (false).

bool CrowdsourcingModels.Results.IsTimeMultimodeModel

get

Flags whether the model instance is a BCC time model (true) or not (false).

Dictionary<string, Discrete> CrowdsourcingModels.Results.LookAheadTrueLabel

getprotected set

The predicted label for each task when doing simulations from the current model state. It avoids overwriting the true label posterior.

Dictionary<string, Dirichlet[]> CrowdsourcingModels.Results.LookAheadWorkerConfusionMatrix

getprotected set

The look-ahead posterior of the confusion matrix of each worker obtained after simulating a new label in look-ahead run mode.

DataMapping CrowdsourcingModels.Results.Mapping

getset

The data mapping.

double [,] CrowdsourcingModels.Results.ModelConfusionMatrix

get

The confusion matrix of the predicted true labels against the gold labels The rows are the gold labels and the columns are the predicted labels.

Bernoulli CrowdsourcingModels.Results.ModelEvidence

getprotected set

Model evidence.

double CrowdsourcingModels.Results.NegativeLogProb

get

The negative log probability density (NLPD) scores of the current true label predictions.

Dictionary<string, int?> CrowdsourcingModels.Results.PredictedLabel

getprotected set

The predicted label for each task

Dictionary<string, Discrete> CrowdsourcingModels.Results.TrueLabel

getprotected set

The posterior of the true label for each task.

Dictionary<string, Discrete> CrowdsourcingModels.Results.TrueLabelConstraint

getprotected set

The posterior for the constraint that allows online learning for the true label variable.

Dictionary<string, Discrete> CrowdsourcingModels.Results.WorkerCommunity

getprotected set

The community membership probabilities of each worker.

Dictionary<string, Dirichlet[]> CrowdsourcingModels.Results.WorkerConfusionMatrix

getprotected set

The posterior of the confusion matrix of each worker.

Dictionary<string, Vector[]> CrowdsourcingModels.Results.WorkerConfusionMatrixMean

getprotected set

The mean of the posterior of the confusion matrix of each worker.

double CrowdsourcingModels.Results.WorkerLabelAccuracy

getprotected set

The accuracy of the worker labels.

Dictionary<string, Dictionary<string, Discrete> > CrowdsourcingModels.Results.WorkerPrediction

getprotected set

The predictive probabilities of the labels produced by each worker.

Dictionary<string, VectorGaussian[]> CrowdsourcingModels.Results.WorkerScoreMatrixConstraint

getprotected set

The posterior for the constraint that allows online learning for worker confusion matrices int the community model.

The documentation for this class was generated from the following file:

C:/Users/Matteo/Source/Repos/active-crowd/CrowdsourcingModels/Results.cs

Classes

Public Member Functions

Static Public Member Functions

Protected Member Functions

Properties

Detailed Description

Member Function Documentation

Property Documentation