Table of Contents
1Scope
2References
3Abbreviations
4Introduction
4.1Tasksandcomponentsofaspokendialoguesystem
4.2Telephoneinteractionwithaspokendialoguesystem
4.3Qualityaspectsandinfluencingfactors
4.4Subjectiveevaluationmethods
5Spokendialoguesystemcharacterization
5.1Agentfactors
5.2Taskfactors
5.3Userfactors
5.4Environmentalfactors
5.5Contextualfactors
6Experimentalset-up
6.1Systemset-upandWizard-of-Ozsimulation
6.2Testscenarios
6.3Testsubjects
7Questionnaires
7.1Questionsrelatedtotheuser'sbackground
7.2Questionsrelatedtotheindividualinteraction
7.3Questionsrelatedtotheuser'soverallimpression
ofthesystem
8Usabilityevaluation
9Analysisandinterpretationofcollectedinformation
AppendixI-Scenarioexamples
BIBLIOGRAPHY Abstract
Describes methods and procedures for conducting subjective evaluation experiments for telephone services which are based on spoken dialogue systems. The respective systems enable a natural interaction via spoken language and possess speech recognition and interpretation, dialogue management, and speech output capabilities.
