ASA PRESSROOM

Acoustical Society of America
134th Meeting Lay Language Papers

[ Lay Language Paper Index ] [ Press Room ]

Subjective Evaluation Of Classical Guitars

Felipe Orduña-Bustamante, felipe@aleph.cinstrum.unam.mx

Ricardo R. Boullosa
Centro de Instrumentos UNAM, Circuito Exterior CU,
México DF, CP 04510, MEXICO.

Popular version of paper 2aMU10
Presented Tuesday morning, December 2, 1997
134th ASA Meeting, San Diego, CA
Embargoed until December 2 1997

Enjoyment of musical performance is affected by many factors which include, among others, the technical and artistic abilities of the musician, the quality of the musical instrument, the room, etc. Some of these aspects, specially with regards to the musical instrument and the room, involve physical properties (mechanical, acoustical, etc.) which can in principle be measured in a very objective and precise way. However, in the end, the enjoyment of music remains by its very nature a purely subjective matter. Something which surely cannot be meaningfully reduced to a series of quantitative measurements. In spite of this, it is still desirable to establish certain correlations between measurable quantities and subjective preferences, if only to provide guidelines for the design and construction of musical instruments and concert halls.

A research program of this kind involves consideration of the following issues:

Subjective evaluation of quality.
Measurement of physical parameters.
Correlation of subjective quality with physical parameters (or quantities derived from them).
Optimization of design and construction.

This is obviously the same general approach guiding the production of many items intended for human consumption (broadly speaking). The approach can be clearly identified in the the field of room acoustics, for example, where physical factors and derived parameters such as reverberation times, lateral reflections, early energy, clarity index, etc. have been identified as subjectively important and used for the evaluation of existing concert halls and for the planning of new ones.

In the case of musical instruments, the research on subjetively important factors has been developed to a somewhat lesser extent, but has already produced very interesting results, specially with respect to the violin, the piano and other instruments. Previous research on the subjective quality of classical guitars has already revealed some degree of correlation with physical characteristics such as atack and decay times of the tones, sound pressure level (related to loudness), certain parameters of the mechanical and acoustical resonances of the guitar, etc. However, physical parameters will no be discussed any further in what folows.

The work described below is a comparative study of two different methods for the evaluation of the subjective quality of a set of classical guitars. One method is based on "blind" listening tests by a general audience and another is based on direct inspection and playing by guitar teachers and students. Both tests were carried out on the same set of four guitars, ranging in quality from "low" to "medium-high" (their prices in USD range from $50 to $500 approximately). In what folows, the guitars are identified by the three-letter codes: CHF, APL, RRB and FOB (named after their owners and listed by price with CHF being the cheapest).

Listening tests

The listening tests were carried out in a small lecture room with a capacity of 120 seats; the guitars were played behind a dark curtain, limiting subjects listening to the guitars to purely acoustical clues as a basis for their evaluation. A short musical fragment, about 10s long, was played in succesion on two of the guitars; the subjects were then given the task of deciding which sound they liked most, that of the first guitar (A), that of the second (B), or none (X). Pairwise comparisons like this were done on a randomized series of all the 12 possible pairings of the 4 guitars (including different orderings of the same pair of guitars) and 4 control pairs in which the same guitar was played twice. This gives a total of 16 pairs using the same musical fragment. Two different musical fragments were used during the tests, these being the first few measures of a Tarantella in A minor by Fernando Sor and of the Prelude in E major from Suite BWV-1006 by Johann Sebastian Bach. Using each of these fragments in the 16 pairs resulted in a total of 32 pairs to be evaluated during the listening tests. These were carried out on different dates in two separate 40 minute sessions in which the 32 pairs were evaluated; the guitars were played by two different players, the same during each session. A total of 51 subjects participated voluntarily during the two sessions (a few of the subjects participated in both). All the subjects consider themselves to have normal hearing and their personal involvement in musical activities range from non-musicians to professional guitarists. The following figure shows how votes were distributed by option (ABX) during the tests

A reasonable low proportion of X-votes was obtained which gives some confidence to the relevance of the results. On the other hand, the proportion of X-votes is not altoghether negligible and may reflect the difficulty of the test which was expressed by some of the subjects. Also interesting is the fact that the proportion of B-votes is clearly shown to be larger than the proportion of A-votes. This seems to suggest that a short-term memory effect is taking place, in which listeners tend to prefer the second guitar (the more recently played) over the first, in spite of the fact that listening to each pair lasts less than about 40s. The following figures show different ways of quantifying the subjective quality calculated as a percentage of the number of votes obtained by each guitar, and where 100% means a guitar that obtained the maximum number of votes possible in each case (this particular way of counting implies that the percent values do not add to 100% in this case).

Differences between the guitars are less clearly seen when they appear first in a pair, which is consistent with the previous observation regarding the bias towards the second guitar in a pair. The label of votes "as A or B" corresponds to percentages of the total number of votes, while "as A and B" corresponds to a special count in which ambiguous votes have been eliminated (a subject emits a pair of ambiguous votes when (s)he prefers different guitars in reversed presentations of the same pair of guitars). There is a significant difference (about 50%) between the total voting and the unambiguous voting which is surely due to the bias towards the second guitar mentioned above. In any case, the unambiguous voting "as A and B" can be taken as the more reliable result from this test.

Playing tests

The playing tests were carried out in a small meeting room. Guitar teachers and students were invited to spend 20 minutes freely and privately inspecting and playing the guitars, and to fill out a form in which they sorted the guitars according to the following aspects: APPEARANCE, TUNING, TIMBRE, VOLUME, COMFORT and QUALITY. A total of 22 guitarists participated in the tests. Each guitarist produced six lists of the guitars, sorted from the best to the worst in each aspect. A subjective rating was calculated by assigning 4 points to first place, 3 points to second place, 2 points to third place and 1 point to last place. The total number of points obtained by each guitar in each aspect was calculated and then normalized to give 100 points for a guitar rated best by all players. The results are shown in the following figures.

The rating of the guitars shows a great consistency among the different criteria. The second figure shows that the combined rating according to the first five aspects, labeled "All aspects", coincides very well with the rating of the overall subjetive quality. The rating of the guitars is also consistent with the results of the listening tests.

Conclusions

The results from the experiments described above reveal that the two types of test produce essentially the same sorting of the guitars by subjective quality (which is also consistent with a sorting according to price). The listening tests, based on pairwise comparisons of guitars playing the same musical fragment, is judged by the subjects to be a difficult task. This is also reflected in the results. Additionally, an interesting effect is consistently observed in which a higher proportion of votes is obtained by the second guitar played in each pair, in spite of the fact that listening to each pair of guitars lasts less than about 40s. This seems to suggest a certain natural limit in the short-term memory abilities of the subjects in recalling the first guitar. A data processing technique is described which produces unbiased results from this type of test. The playing tests, on the other hand, are judged by the players to be a comparatively easy task and the results are also more easily interpreted.

These observations will have to be confirmed by more extensive tests with the participation of more subjects and with instruments of higher quality. On the other hand, physical tests carried out on the same guitars will have to be correlated with subjective preferences in order to establish the precise mechanical and acoustical characteristics that define a fine classical guitar. Overall, the results of our tests fit nicely with each other, lead to reasonable conclusions and, in particular, suggest that the old wisdom: "ask the expert" seems to be firmly grounded.