#! python """A script to run a reading experiment. Text stimulus, verbal response. It puts text up on a screen, the subject reads it, and the program records what the subject says. It is designed for recording largish blocks of text (paragraph sized) but is flexible and automated. Usage ===== Run it as:: collect_data -d some_directory groupname and it will read I{some_directory}C{/stimuli/}I{groupname}C{.fiat} display stimuli, record speech, and write I{some_directory}C{/response/}I{groupname/datecode}C{.fiat} that contains all the input stimuli, along with metadata. It also produces one .wav file per utterance, named I{some_directory}C{/response/}I{groupname/datecode/StimulusNumber_RepetitionNumber}C{.wav} These are Fiat 1.2 format data files, and can be read with L{gmisclib.fiatio} from the U{Sourceforge} U{speechresearch project}. The Fiat format is defined at U{http://www.phon.ox.ac.uk/files/pdfs/fiat.pdf}. Features ======== - Source code is available, so it can be modified as necessary. - Written with in Python using the widely available GTK package for ease of installation. - Instructions to the subject and the texts the subject reads can be written in any Unicode characters, so the software can be used for most languages. - Customizeable via a file to defines the experiment. - Yields a file of metadata describing exactly what went on. - Suitable for reading paragraph-sized chunks of text. Operation ========= The software can be conveniently run if the experimenter has the keyboard and the subject has the mouse. In the body of the experiment, the subject clicks "next" to see a prompt, then speaks. Then, the experimenter types "q" to terminate the recording. The software then pauses, waiting for the subject to click "Next" (to go on to the next stimulus) or "Repeat" to read the stimulus again. Alternatively, the experimenter can hit the space bar to go forward, or the "r" key to repeat a reading. Note that the software can enforce a limit on how many times a text can be read, via the C{Maxreps} value. If the experimenter types 's', the recording is terminated and deleted, effectively skipping that stimulus. A comment is left in the output file, but no other metadata for this utterance. The experimenter can also type "x" -- this acts like "q", but leaves a mark in the "flag" column. The software records one audio .wav file for each line in the control file, and writes one line in the output metadata file. In typical operation, the I{groupname} selects which experimental group a given subject is in. A subject is then simply identified by the filename of the output metadata file, so the data is naturally anonymized. However, if a pre-existing subject ID needs to be carried through, or if a single subject comes in for more than one session, then the easiest solution is to use a different I{groupname} for each subject. Typically, you'd use the subject ID number as the groupname, and simply make a copy of the input (control) file for each subject in a group. Control File Format =================== The program reads a file (also in Fiat format) that controls many aspects of the experiment (on a stimulus-by-stimulus basis if need be). The variables below affect the experiment; any other data in the input file is simply passed to the output metadata file. Values can be set in the header of the input Fiat file (in which case they have effect throughout the experiment) and/or they can be given columns of their own. If they have a column of their own, and a value is present then that value over-rides the default. (Note that the code C{%na} in a column means that no value is given, so the value specified in the header, if any, would be used.) To say this again: the values set in the header and the columns of data in the input file are merged together. When the program is looking for a value, it looks first in the column data, then if nothing is found, it takes the value from the header. The software has three text areas. A small one above for instructions to the subject, another small one above for status information, and a big one below for material to read. - INSTR_* : Instructions to be given to the subject at various points in the experiment. The instructions appear in the upper area. - C{INSTR_key_for_first} : Show this just before the first stimulus is presented. - C{INSTR_last_chance} : Warn the subject that this is their last try for this stimulus. - C{INSTR_continue_repeat} : Ask wheter to continue to the next stimulus or repeat the last one? - C{INSTR_continue_norepeat} : Much like continue repeat, except this is presented on the last stimulus, when there is nothing more to do, but you could still try the final stimulus again. - C{INSTR_read} : "Please read the text below" or similar. - C{INSTR_welcome} : An instruction to present at the beginning of the experiment. (E.g. "Welcome") - C{INSTR_thanks} : An instruction to present at the end of the experiment (E.g. "Thank you") - C{B_repeat} : The text of the "Repeat stimulus" button - C{B_next} : The text of the "Next stimulus" button - C{STAT_recording} : What to put in the "status" box when the recorder is running for the first time on a stimulus - C{STAT_repeating} : What to put in the "status" box when the recorder is running on a repeat of a stimulus. - C{Maxreps} : How many times can you repeat a stimulus? - C{BigInfo} : What to put on the main screen while the subject is waiting. (This is typically blank -- it is a way to emphasize unusual instructions.) Output (Metadata) File Information ================================== All the input control information is copied to the output metadata file. Additional columns are added as follows: - C{stimulusTime1} : A moment shortly before the stimulus is visible. - C{stimulusTime2} : A moment shortly after the stimulus is visible. - C{d} : A directory containing data, relative to the directory that holds the metadata file. - C{f} : The root of a particular utterance's audio file, within C{d}. C{d} and C{f} are used together, so the path to the audio file is C{d/f.wav}, starting at the directory that holds the output metadata file. - C{recordStartTime1} : A moment shortly before the recording starts. - C{recordStartTime2} : A moment shortly after the recording process has been forked. Unfortunately, we do not know if the recording has started yet or not, but at least the recording process has been created. On modern Linux systems (c2009) the recording starts no more than 50ms after recordStartTime1. - C{RecordEndTime1} : A moment shortly after the recording program has shut down. - C{i} : A integer count of which utterance, 0...N. - C{rep} : An integer count of how many times the subject has attempted this utterance. - C{flag} : Zero, or one (if the 'x' key was pressed during the utterance). Software downloads should also be available from the "speechresearch" project on http://sourceforge.org, the Oxford University Library system, and http://kochanski.org/gpk . This software is copyright Greg Kochanski (2010) and is available under the Lesser Gnu Public License, version 3 or higher. It was funded by the UK's Economic and Social Research Council under project RES-062-23-1323. This is available from http://sourceforge.org/projects/speechresearch, http://kochanski.org/gpk/papers/2010/aesop_data_collect, and http://www.phon.ox.ac.uk/files/releases/2008aesopus2_data_collect.tar @copyright: Greg Kochanski, 2010 @license: Gnu Public License, version 3 or higher. @contact: gpk@kochanski.org @contact: greg.kochanski@phon.ox.ac.uk @version: Aesop.0.20.2 @note: Please cite in academic papers as "data collection software used in "Rhythm measures with language-independent segmentation", Anastassia Loukina, Greg Kochanski, Chilin Shih, Elinor Keane and Ian Watson Proceedings of the 10th Annual Conference of the International Speech Communication Association (Interspeech 2009). ISSN 1990-9772 Brighton, UK, 7--10 September 2009, pp 1531--1534. The software may be downloaded from http://www.phon.ox.ac.uk/files/releases/2008aesopus2_data_collect.tar . (URL checked ZZZ/ZZZ/ZZZ.) """ import os import signal import datetime import subprocess from gmisclib import fiatio from gmisclib import gpkmisc from gmisclib import die import exp_collection as EC import gtk import gobject ROOT = '/projects/aesop/data_files' TEXT_ROOT = '/projects/aesop/Texts_for_recording' RATE = 16000 CHANNELS = 2 LMARGIN = 100 # Blank margin to left of stimulus area RMARGIN = 100 # Blank margin to right of stimulus area CID_R = 1 class autokilled_process(object): """This class represents a subprocess that's automatically started and automatically killed by C{__del__} or an explicit call to C{close}(). """ def __init__(self, args, **kv): self.p = subprocess.Popen(args, **kv) def __del__(self): self.close() def close(self): if self.p.poll() is None: os.kill(self.p.pid, signal.SIGINT) self.p.wait() return self.p.returncode class gui(EC.GUI_base): """This is the Graphical User Interface for the experimental data collection software. @note: all the S_* functions represent states during the data collection process. The program hops from one to the other, around in a loop through the S_* functions for each utterance. """ def set_button_texts(self, rs, ns, repeat=None, next=None): """The GUI has two buttons for the subject to press. One moves on to the next paragraph to read; the other repeats the current paragraph. @param rs: Should the 'repeat' key accept clicks? @type rs: L{bool} @param ns: Should the 'next' key accept clicks? @type ns: L{bool} @param repeat: A label for the "repeat" button @type repeat: str or None @param next: A label for the "next" button @type next: str or None """ if repeat is not None: self._repeat.set_label(repeat) if next is not None: self._next.set_label(next) self._repeat.set_sensitive(rs) if rs: # KLUGE self._repeat.hide() self._repeat.show() self._next.set_sensitive(ns) if ns: # KLUGE self._next.hide() self._next.show() def __init__(self, extra_line_space=5, extra_para_space=5, top_font=None, stim_font=None): """Creates an instance of the GUI. (Normally there is just one.) @param extra_line_space: How many extra pixels should separate one line of the stimulus from the next? @type extra_line_space: L{int}, in pixels @param extra_para_space: How many extra pixels should separate one paragraph of the stimulus from the next? @type extra_para_space: L{int}, in pixels @param top_font: The name of the font used in the top section of the GUI. @param stim_font: The name of the font used to present the stimulus @type stim_font: L{str}, passed into L{pango.FontDescription} @type top_font: L{str}, passed into L{pango.FontDescription} """ self._next = None self._repeat = None EC.GUI_base.__init__(self, extra_line_space=extra_line_space, extra_para_space=extra_para_space, top_font=top_font, stim_font=stim_font) def connect_experiment(self, expcall, log): """Connect the GUI to the class that defines the experiment. @param log: A place to log everything that happens in the experiment @type log: normally an instance of L{fiatio.merged_writer} @type expcall: a function pointer @param expcall: a function that knows how to change the experiment's state in response to keyboard and mouse events. """ # We need to stick in the extra "None" in these # calls to substitute for the "event" argument # that isn't supplied by Button clicks, but is # supplied by keyboard keypress events. self._repeat.connect("clicked", expcall, None, log) self._next.connect("clicked", expcall, None, log) EC.GUI_base.connect_experiment(self, expcall, log) class experiment_c(EC.experiment_base): """A class that defines the sequence of the experiment. """ def __init__(self, hdrs, stimlist, log, outname): # You can set paragraph and line spacing and fonts # by adding extra arguments to the GUI_base constructor. self.gui = gui(top_font="Serif 12", stim_font="Serif 24") self.gui._stim.set_left_margin(LMARGIN) self.gui._stim.set_right_margin(RMARGIN) EC.experiment_base.__init__(self, stimlist, hdrs, self.gui) self.info = {} self.p = None self.repcount = 0 self.gui.connect_experiment(self.event, log) self.status = '-' self.outname = outname def instruct(self, s): """Give an instruction to the subject. """ text = self.get('INSTR_%s' % s) self.gui.instr_win().get_buffer().set_text(text) def present(self, s): """Present a stimulus to the subject. """ if 'textfile' in s and s['textfile']: text = EC.get_text(os.path.join(self.hdr['TEXT_ROOT'], s['textfile'])) elif s['text'].startswith('@'): text = EC.get_text(os.path.join(self.hdr['TEXT_ROOT'], s['text'][1:])) else: text = s['text'] self.info['stimulusTime1'] = datetime.datetime.now().isoformat() self.gui.stim_win().get_buffer().set_text(text) gobject.idle_add(self.set_timing_info, 'stimulusTime2') def clear_stimulus(self): self.gui.stim_win().get_buffer().set_text('') def set_timing_info(self, name): self.gui.window.get_screen().get_display().sync() # Ensure that the display is actually # showing the prompt. self.info[name] = datetime.datetime.now().isoformat() return False def get_audio_file_name(self): self.info['d'] = self.outname self.info['f'] = "%06d_%1d" % (self.i, self.repcount) return os.path.join(self.info['d'], self.info['f']) + '.wav' def start_recorder(self): audiofilename = self.get_audio_file_name() gpkmisc.makedirs(os.path.dirname(audiofilename)) args = ['arecord', '-t', 'wav', '-f', 'S16_LE', '-r', str(RATE), '-c', str(CHANNELS), audiofilename ] print 'subprocess', args self.info['recordStartTime1'] = datetime.datetime.now().isoformat() self.p = autokilled_process(args) self.info['recordStartTime2'] = datetime.datetime.now().isoformat() return audiofilename def ok_cont(self): return not self.is_last_stimulus() def ok_rep(self): return self.i>=0 and self.repcount= int(self.get('Maxreps')): self.instruct('continue_norepeat') else: self.instruct('continue_repeat') self.gui.set_button_texts(self.ok_rep(), True, self.get('B_repeat'), self.get('B_next')) biginfo = self.get('BigInfo', '') if biginfo: if biginfo.startswith('@'): # print '@BigInfo from', self.hdr['TEXT_ROOT'], biginfo[1:] biginfo = EC.get_text(os.path.join(self.hdr['TEXT_ROOT'], biginfo[1:])) # print '@BigInfo got', biginfo self.gui.stim_win().get_buffer().set_text(biginfo) # print 'S_waiting', ev if ev is self.gui._next or ev == ' ': self.clear_stimulus() return self.S_continue elif ev is self.gui._repeat or ev == 'r': self.clear_stimulus() return self.S_repeat return None def S_continue(self, ev, log): # print 'S_continue', ev if self.ok_cont(): self.repcount = 0 self.next_stimulus() self.status = self.get('STAT_recording') return self.S_present # print '-> S_final' return self.S_final def S_repeat(self, ev, log): # print 'S_repeat', ev if self.ok_rep(): self.repcount += 1 self.status = self.get('STAT_repeating') return self.S_present return self.S_waiting def S_present(self, ev, log): # print 'S_present', ev self.instruct('read') s = self.get_current_stimulus() assert s, "Empty stimulus" self.info = s audiofilename = self.start_recorder() self.gui.status_push(CID_R, '%s %s' % (self.status, os.path.basename(audiofilename))) self.gui.set_button_texts(False, False, None, None) self.present(s) self.info['i'] = self.i self.info['rep'] = self.repcount return self.S_recording def S_recording(self, ev, log): # print "S_recording", ev if isinstance(ev, str) and (ev==u'q' or ev==u'x'): iret = self.p.close() self.info['RecordEndTime1'] = datetime.datetime.now().isoformat() assert iret in [0,1], "Bad return code from arecord: %d." % iret EC.check_wav(self.get_audio_file_name(), self.gui) self.p = None if ev == u'x': self.info['flag'] = 1 else: self.info['flag'] = 0 self.gui.status_pop(CID_R) self.clear_stimulus() log.datum(self.info) log.flush() if not self.ok_cont() and not self.ok_rep(): return self.S_final return self.S_waiting elif isinstance(ev, str) and ev==u's': iret = self.p.close() os.remove(self.get_audio_file_name()) self.p = None self.gui.status_pop(CID_R) self.clear_stimulus() log.comment('Skipped') log.flush() if not self.ok_cont() and not self.ok_rep(): return self.S_final return self.S_waiting return None def S_initial(self, ev, log): if self.first_entry(): self.instruct('welcome') self.gui.set_button_texts(False, True, self.get('B_repeat'), self.get('B_next')) self.gui.status_push(CID_R, 'Waiting.') if ev is self.gui._next or isinstance(ev, str): return self.S_waiting return None def S_final(self, ev, log): # print "S_final" if self.first_entry(): self.instruct('thanks') gobject.timeout_add(2000, self.gui.destroy, None) return None def run(argv): global ROOT, TEXT_ROOT arglist = argv[1:] while arglist and arglist[0].startswith('-'): arg = arglist.pop(0) if arg == '--': break elif arg == '-d': ROOT = arglist.pop(0) elif arg == '-t': TEXT_ROOT = arglist.pop(0) else: die.die("Unrecognized flag: %s" % arg) try: subjectID = arglist[0] except IndexError: die.die("Need to specify subject ID") datecode = datetime.datetime.now().strftime('%y%m%dT%H%M') d, c = fiatio.read_merged(open(os.path.join(ROOT, "stimuli", subjectID) + ".fiat", 'r') ) h = {'subjectID': subjectID, 'ROOT': ROOT, 'TEXT_ROOT': TEXT_ROOT} outname = os.path.join(ROOT, "response", subjectID, datecode) print 'LOG file=', outname gpkmisc.makedirs(os.path.dirname(outname)) log = fiatio.merged_writer(open(outname + '.fiat', 'w')) die.info("Writing output to %s" % outname) experiment = experiment_c(h, d, log, outname) try: log.headers(experiment.get_hdrs()) log.header('Start', datetime.datetime.now().isoformat()) experiment.gui.status_push(1, "Log file = %s" % outname) experiment.gui.main() log.header('End', datetime.datetime.now().isoformat()) log.close() finally: experiment.close() if __name__ == '__main__': import sys run(sys.argv)