Some Additional Thoughts on Illich: Responsibly Limited Tools [ #captions]

On the Path to Convivial #Captioning

When considering #captioning as a convivial tool—or at leas potentially as a tool to help foster, if only temporarily, a more convivial society or culture, it’s useful to fill in some other pieces of Illich’s view.

For Illich, a convivial society also places technology with a specific role:

“modern technologies serve politically interrelated individuals rather than managers” (1973, xii).

The emphasis is clearly on individuals and citizens communicating and connecting with each other as opposed to giving supervisory or surveillance powers to the bosses. Placing #captions in this frame would then have people captioning text for each other—as service, for pleasure, for fun, for critical thinking and engagement—as opposed to captioning as product or control.

Illich does not allow technology anything close to free reign—certainly not the way technology is currently allowed to drive cultures and government. Instead, he uses convivial to “Label a culture of responsibly limited tools” (1973, xii-xiii). Keywords: responsibly limited tools. How might captioning fit here? Are there ways in which captioning could be used irresponsibly—so much so that people are actually harmed or damaged from the process? It’s hard for me to conceive of right now, but perhaps if captions were intentionally created that misrepresented what was said in the piece? Or? Can you think of a reason why captions **should** be limited?

Illich makes it pretty clear that in a convivial society, every citizen has free and ready access to the community’s tools. This access and use is limited only when it might impinge on another’s freedom. One way to interpret this within the frame of captioning is that the tools used for caption creation, rendering, and viewing should be free and/or open source. [This is another discussion entirely, free vs. open source, but either solution might work for convivial captions.]

What do you think?

Conviviality & #Captions: A Wandering for #digiwrimo

Opening & Background

#Captions are on my mind, and I’m working to integrate captions and different approaches to captions within the larger frame and work I have engaged in and researched. Connecting the content to what I’m involved with, with what I care about, is a basic approach that relates to constructivist learning. A part of constructivism, as well, is connecting with others and learning with them. I watch, read, review, and consider others’ work on captioning—a form of social learning. End goal, on top of all of this, is to move into constructionism and engage in intelligent, theory-driven practice. 

a light caveat

[I realize this might seem in part, like a surface or shallow discussion, but one of the roles of a blog is to reflect, discuss, and explore lines of flight instead of always trying to remain deep and focused. Those have their value, too, but my use of this blog is to explore, connect, reflect.]


Now Back to Illich

I’ve been reading about conviviality slowly and surely, thinking about it as well, since I first learned of it in 2005. A clear definition of conviviality comes from Illich’s Tools for Conviviality reads:

"autonomous and creative intercourse among persons, and the intercourse of persons with their environment" (1973, p. 10).

That leaves a lot of room for interpretation and adaptation. Key ideas here are that interactions are not moderated or controlled by hierarchy or external authority. Instead, people communicate directly with and among themselves. Similarly, their is no moderator between people and the environment around them.

a second caveat

It’s important to note here that I am probably veering away from Illich’s compelling and holistic vision for a society. Illich was not writing for short-term solutions or “fixing” capitalism or industrial society. No. He wanted to control technology, to make sure that the health of people and the planet’s welfare came first; any technology that would disrupt, destroy, or challenge that would, and should, be banned or stopped by an intelligent culture. Why, after all, would you allow the development of tools that destroy and hurt the planet or other human beings?

A short version of Illich’s vision can be found in his description of a convivial society:

“A postindustrial society must and can be so constructed that no one person’s ability to express him- or herself in work will require as a condition the enforced labor or the enforced learning or the enforced consumption of another” (1973, p. 13).

In this work, in my thinking, I am thinking about how captions might be a *convivial technology*. I don’t claim, by any stretch, that the captions could remake society or other such delusions.

However, if we frame or consider captions and captioning as a convivial technology, what does that do to captions? How might that reframe practice and application? What might evolve out of this? Is it possible that a small community of captioners might be able to create a Temporary Autonomous Zone (TAZ)?


Remembering the Context; Captioning as Literacy

This may seem far afield from how we traditionally see and use captions, and I’m fine with that. I’m not attempting to overthrow the traditional use, application, or framework of captions. That would be silly. Captions, as they currently exist, do fulfill a role—although they often do it to a limited extent.

Currently, it appears as though captioning is very limited in its approach and application. Multiple scholars and practitioners have pointed this out (I can name drop later when the literature review is more thorough, but trust me on this). You could also frame captioning as a form of literacy. Wish this idea was mine, but from what I can tell it emerged in the work of Brueggemann and Karl Fredal in their “Captioning CODA” portion of [Articulating Betweenity].

One metaphor is that captioning is a brush. So far that brush has only been used to write letters. Calligraphy can be beautiful, no question:

However, the experiences of reading calligraphic text, and learning to create it, are very different from reading illuminated text, and learning to create them. 

Illuminated texts rely upon, and need, the alphabet, the brushes, etc. Illuminated texts are built upon the basic, pragmatic communication tools. Why should we remain with those basic communication tools? 


Back to Conviviality

As captioning is currently practiced, there is usually only one captioned track—the track paid and/or provided for by the media creator. Maybe this is the master narrative, but captions often lack in emotional and connotative aspects (see Zdenek and Snell—more details in a future article).

Captioning convivially could have a very different approach, an approach where the captioning is not “paid” for by the media company or government, by some top-down authority, but instead horizontally. Sure, yes, I might want to take the original “master” narrative from the media company—but then modify the captions, indicate sounds that I know are important (like the sound of a click on the phone in an early “X-Files” episode—the sound that indicates a phone tap but which is not captioned) in my viewing and reading of a show or series. If I had friends, local or global, who liked the same shows, they might create their own captions that center on the sounds and content which interest them—and maybe add some animated captions here and avatars for speaker identification.

In many cases, many of our captions might have the same core body of text—especially in terms of dialogue—but there would be variations, iterations, and versions. The captions are created for each other—for other fans and viewers—as opposed to being a product that is bought, sold, and consumed.


Illich’s Environment

Given Illich’s critique of technology and its manifold abuses, I think he would disagree with the following discussion. Perhaps. 

Let’s return, briefly, to Illich’s definition of conviviality:

“autonomous and creative intercourse among persons, and the intercourse of persons with their environment (1973, p. 10).“ 

What if this environment is not just the physical environment but the mediascape in which we interact and live? Is that a potential reading of Illich’s discussion? 


What do you think? Am I stretching things a bit too far? Do you want more evidence or proof? What directions might things go?



Calligraphy image

Calligraphy - calligraphie


Illuminated Mansuscript

Illuminated Manuscript, Duke Albrecht's Table of Christian Faith (Winter Part), The Veil of Veronica, Walters Art Museum Ms. W.171, fol. 1v


Serious Tech Frustrations on the Captioning Research Path

First Fun: Windows on VMware

For the past ninety minutes, I have had some serious technology frustrations. First, when I try to grab some software to use on my VMware (running on my Mac), it says that almost all 60 gigs of the memory are taken up.

Taken up with what? Considering I’ve used the VM, oh, 10 times, I have not brought in SIXTY GIGS OF FILES. So, yeah, pretty much any thoughts I was having about the convenience of working with VMware on my Mac are now dying a quick death. Why?

I don’t want to waste time, and this has been irritating.

So I used WinDirStat from WinDirStat.info and found that nearly 27 Gig was being up by Windows 7. That seemed a bit nuts. Another 20 Gig eaten up by Dropbox—even though I was pretty sure I had removed DropBox.

And then I discovered that I could reconfigure the virtual disk—so I’m doing that—and adding about 240 more Gig of memory. Sure hope that works. Fortunately, the iMac has 1T of drive.


Second Fun: 2009 HP Laptop 

In order to work with some video decryption for caption analysis, I thought I could break out my HP—the laptop that served me during my dissertation years. Well, I should have known better.

First, it crashed with blue screens I could not read in time—twice. Then there were multiple window uploads. Then there were failures to connect to the internet. Then there’s the reality of working with a 3GB processor machine when I’m used to 8 or 32… I feel your pity. So any errors it makes are slow in the happening.


Third Fun: Working with DVD decryption

In the midst of all this, I’m attempting to understand the DVD decryption process so that I can grab the captions and run analyses on the content. All of this work that I have been doing has been about one basic problem: making sure a machine is capable of downloading and handling the decryption software. Thus far, it has not happened.

It’s incredibly difficult to explain the irony, and frustration, that comes from having multiple machines—machines with different OS’s and from different periods—and yet I can’t seem to get from point A to B.

Oh, yes, and in the midst of all this, I’ve kind of lost track of why I was doing all this work: to access and write about captions. But now, now it just feels entirely and incredibly frustrating with the tech.

The rich irony is that this process should be nearly dead simple. But it’s not—not for me at least.


So...

So I’m stepping away, going to go exercise, and then I’m going to write and read about captions. I’m done messing about with the hardware, software, and downloads. After over two hours of mucking about, I’m done.

Missing Heartbeat: Non Speech Information or Mood Creator?

While researching captions and captioning, I came across the work of Brenda Jo Brueggemann. I then found her video, Why I Mind, on YouTube. Short, intense, focused, and engaging. However, I encountered a component that interested me: from minute 2:00 to 2:30 in the background a heartbeat can be heard in the soundtrack while describing a tense situation. The heartbeat works to increase the feeling of tension, anxiety--especially because it ends when she describes the death of a D/deaf student.

However, there is no indication of the heartbeat in the captions.

The heartbeat does not seem to be non-speech information, from what I can tell, as Zdenek describes it: NSI "includes sound descriptions, speaker IDs, manners of speaking, music lyrics, and any other information that might be needed to convey a full understanding of the sound track." The heartbeat has nothing to do directly with anything depicted in the images or the speech on the video. However, the heartbeat is there and it directly impacts the viewer and potentially their experience of the text. Perhaps it could be considered akin to a musical soundtrack? If we look at Snell (2012), this omission is important:

“Closed captioning’s success ... lies in its ability to assist the viewer in not only understanding the denotative meaning being expressed in the spoken dialogue of a captioned piece, but also more importantly by captioning’s proficiency in facilitating in its users emotional experiences that coincide with those suggested by and made available through the mix of spoken dialogue, music, and the visual action of the captioned media.” 

“Such an understanding of closed captioning departs from general conceptions and studies of it that define its effectiveness and efficiency as being nearly exclusively contingent upon logo-centric variables. That is, the previous conception of it as not only assisting the viewer in understanding the denotative meaning being expressed in the spoken dialogue via its textual translation but also as enabling emotional experiences, departs from the traditional view of captioning’s success as being dependent on language –based variables that enable denotative comprehension alone” (emphasis added; Snell, 3 2012).

If we want captions to connect with the viewer, the reader, and to fully convey a level of connection that Hearing viewers have, if one of captioning's key goals is to provide equivalent experiences--at least so far as possible--for those hearing the soundtrack to videos and those who are D/deaf (and Snell asserts this reading of U.S. Public Law (PL) 85- 905 and PL 87-715 on pages 5-6 in her dissertation), then something seems to be missing. When considering the ethos and pathos that Brueggeman builds in the piece, the heartbeats clearly add on value and weight to the text.

This is situation is interesting, important, and compelling. It is exactly this type of situation that could benefit from additional or new approaches to captioning. Perhaps a caption could have read, "sound of heart beat" or "heart beating in background." Perhaps the screen could have pulsed along in time with the heart beat. Perhaps the captioned text could have pulsed. Or perhaps something not mentioned here.

My purpose is not to be difficult or to pick on an independently produced video. Well-captioned videos are hard enough to find, much less an indie video with good captions. However, I wonder if it is not with these smaller videos, with the independent producers and creators, that we can test out new approaches to captioning, that we can experiment with new technologies or try out innovations. 

A final note: I literally just stumbled across this video, and it happened to have a gap that I've been reading, writing, and thinking a lot about lately. This piece was selected for no other reason than that.




Research: No Really, It IS a Research Project (#captions)

Tired of Fail Focus

As several of my earlier posts indicate, I'm pretty focused on #captions at the moment. Nice to be back in the fold caring about something that is interesting. I'm still working through potential topics. I definitely like the close reading approach: examining patterns in captioning. In contrast, I don't feel that just identifying captioning fails is really the way to go. Not for me. I can see the value, sure, but it feels like an approach that does not work well in teaching writing or composition: only identifying patterns of error.

Writing Analogy

When teaching writing, when learning writing, you don't want to just know where things stink. Of course it helps a bit, but if all you hear is, "Fail. Fail. Fail," that gets old pretty quick. Yes, you might want to change it, but it's all negative. All stick, no carrot. No direction is given on which way to go or how to roll. Of course we can do the "Here are three strengths; here are three areas for improvement approach." That worked pretty well--it balances things out, at least in terms of writing.

Wealthy Conglomerates are Cheap

When looking at captioning, though, I'm not sure that offering this kind of suggestion or feedback is going to have much, if any, impact on the captioning industry itself. Come on, if HBO is not willing to pay to have their scrolling captions changed to decent captions--much less get the timing consistent--then I'm not sure that any arguments about aesthetics or best practices is going to change the economic approach to captioning by the entertainment industry. Let me correct myself: I believe that the professional captioners who do the captioning would LOVE to do proper captioning, timing, etc., and have the most effective captions possible. That, however, costs money. From what I can tell, the entertainment industry as well as the entertainment delivery system, pay as little as possible--in most cases--to deliver the minimal legal requirement.

Example 1: Just look at this mess.

Oliver is brilliant, funny, and engaging. Yet HBOnow is presenting this show's captions if it is live. It's not live--it's a recording! You have the transcripts! Can it be that hard to go through and actually caption this properly? It won't cost that much money--for real.

So, considering that the entertainment industry and its deliverers seem to have little interest in doing more than the least possible (if you know otherwise, I'd love to hear about exemplary groups that support top notch captioning--truly!), suggestions about best practices and ways to innovate need not be directed towards industry.

Innovate Where It Matters

Suggestions about innovation, change, best practices, and so forth should be directed towards the captioning community--people who love and care about captioning--as well as citizens and artists who are interested in the interplay of text with image, who like working with pop culture materials, and who might want to try something different. Doubt there's anything new or revolutionary stated here, but I still feel like it needs to be written.

In order to offer meaningful suggestions and feedback, you have to care. You have to be invested. It's hard to be really upset about something like captions in a show if you don't care about the show. Why would you watch a program if you don't want to be informed, entertained, or somehow emotionally connected--even if that emotion is escapism or numbing? To do those things, you need clear messages from the captions; you need captions that not only provide an equivalent experience--you should have captions that expand experience beyond what the show is without captions.

That is ideal captioning. Facilitating that is the challenge. Support and drive for that is rarely going to come from big industry.

What we can do, though, as individuals, as people who like captions and pop culture media, we can run close analyses on texts, we can generate alternatives, we can speculate and build alternative caption models--and all of this falls under the protection of scholarship and Fair Use (assuming you actually transform the work, and why else would you bother investing that much time unless it was transformative?).

But we still have to care.

I know what I care about. I care about, and become invested in, several types of shows. First, the Inspector Morse trilogy: Inspector Morse, Inspector Lewis, and Endeavour. There are plenty of similar shows that are great, too, like Miss Fisher's Murder Mysteries

A second genre are conspiracy thrillers, often with female leads. Alias, Orphan Black, and Dollhouse. Put in a dystopian theme, and I'm all over that.

I used to get into more police shows, but there's too much violence in them. Before my tolerance for violence dropped through the floor, I really enjoyed Rome and The Wire. I still like The Wire, but I can only watch so much before I have to turn it off. I'm at the point where I go looking for specific episodes or specific scenes with characters. I've watched all five seasons twice, and I'm on my third time through. So, yes, I like to watch Bunk at his finest and Omar ripping dealers.

As a person who lives in his small world a bit too much, I was at a bit of a loss as to what other shows fall in related genres. Yes, I can trust Amazon and Netflix; however, past couple months, I spend time looking for shows only to end up watching shows I know for the third time. Maybe I just need something comforting? Or maybe I just don't know what to watch.

Now, though, getting back into captions has me thinking: what body of shows can I analyze? What themes am I going to look for? What types of content do I want to explore?

"Heck ya," said I to myself, "this is the perfect way to be entertained and do scholarship." So I posted a request on Facebook to my peeps. "Peeps," I said, "Hook me up."

Actually, here's what I said:

The responses were great! Here they are:

  • Fringe
  • Buffy
  • X-Files
  • Prime Suspect
  • Covert Affairs
  • Agent Carter
  • Damages
  • The 100
  • Battlestar Galactica (reboot)
  • Dark Angel
  • Sense8

What this means is that I now get to go spend some quality time looking into shows--checking them out--and seeing which ones I end up caring about and being invested in. If I'm going to watch and read the shows, and focus some serious engagement and analysis--not to mention reading the dialogue between two to five times for each show that I look at (if not more times), the show better be good. There had better be something there.

While it might sound strange, one lens I have on this approach is akin to fan fiction. By engaging and investing in the show, the character, the narrative, you get a strong sense of the story, setting, and context. Having this experience, perspective, and understanding--authentic engagement--helps frame where, how, why, and when captions could be tweaked or changed or improved in order to support the overall story and experience.

Obviously my thinking is still developing, and clearly my experience is still pretty limited. However, after years of watching captions and thinking about what I'm watching, this field just seems so exciting, ripe, and engaging--ideas are going in all directions. It will be interesting to watch and track where things settle.

In the mean time, while some of this settles, I plan to keep focused on my Orphan Black analysis, reading of related scholarship, and working with After Effects.


Voice Driven Type Design: Useful for #Captions?

Fascinating article by Woelfel, Schlippe, and Stitz. If I understood it properly, different qualities of a an audio file or spoken voice could be interpreted and impact how the words spoken would be represented typographically. In other words, volume might impact size of the type, font used, leading, if the type was ragged, or what have you.

In terms of captioning, I thought this might be interesting in terms of potentially providing an algorithmic approach for caption generation. Might be a good way to show anger, calm, seduction, etc. Then again, this might also require viewers to learn an entirely new vocabulary or develop an additional literacy to interpret what specific things, like size, font, or color, might mean.

Then there's are a few potential problems: would each company potentially have its own algorithm and thus have no consistent standard? Would the standard be free and given away? Would there be a charge for it? Could this only appear in videos or films that could afford it?

One approach that I think might work is, of course, inspired by the epic and experimental captioning done inNight Watch and the new Sherlock Holmes. Rather than trying to do these kinds of effects on all text, perhaps these could be used for such things like representing NSI (non-speech information) on screen and single word utterances, such as profanities and exclamations. Thus they would be visually different from the normal presentation of utterance and sounds, and yet they could also embody certain components of the utterances in their type design.

Plenty to think about from this piece. Hope that we see more of this type of work--and maybe some of it will enter the realm of captions. Yes!

Considering #Captions in Orphan Black: Some ideas and sources flowing towards speculative captioning

The Personal

I love Orphan Black. I am invested viewer, and I've watched all three seasons twice already. I'm now on my third run through the series. Why? Because it has all the components I enjoy: conspiracy, dystopian near-future, strong characters, an interesting gay male (Fe), a strong female lead (multiple leads, in my view, all played by the same actress). During each viewing, I've watched with the captions on. Most of the time I'm reading and watching the action. It's not just passive viewing; there's active reading as well. Just feels right.

The Trigger

Now that I'm back to working and thinking about #captions--it's been non-stop the past 36 hours--I've been thinking a lot about Orphan Black. I was just watching "Nature Under Constraint and Vexed," (Season 2, Episode 1) and had a sudden revelation--or delusion--about how identity was not clear when Sarah or Alison were impersonating each other. These impersonations are often only clear by hearing the alternation between voices--something that is not clear in the captions [I'm working on a longer piece about this, actually, so I don't want to develop everything here]. That got me thinking.

A lot.

The Action

This morning, then, I decided to track down some scholarship on OB. [Tangent: Am I the only one that finds the initials of the show ironic, especially considering the strong themes about fertility, biopower, birth, fertilization, fertility, and women controlling their bodies? Hmmm.] Anyhow, I found a great dissertation (completed in in 2015): Renegotiating the Heroine: Postfeminism on the Speculative Screen. Opening chapter is pretty clear work on the purpose, etc. Nice work on post-feminism--certainly helped remind me of how little I know in that area and where I need to read. However, somehow I retained focus. Skimmed much dissertating and ended up in the chapter I wanted most: "Just One, I'm a Few": Cyborgs, Reproductive Rights, and Difference inOrphan Black. If you're hoping for some Deleuze, he's not here. Foucault and Biopower are, though. There is, however, plenty of Haraway along with de Laurentis and a bit of Greer.

As I approached this text, my core goal was to uncover other critical perspectives and views of OB. I found that. A bit of discussion on multiplicity--hooray--with Haraway and some connections with realist dystopian fiction. Also cool. Helped me look at the series from a different angle. These angles are important to me because, based on my conversations with Sean, I want to explore what emerged as "speculative captioning" in our conversation. I'm sure that one or both of us, if not others, will work on developing this idea more fully, but the short gist of it (for now at least) is: how we can explore captions' use and applications to open up new perspectives, to try new approaches, to fully employ current technologies and explore captions' aesthetic, political, and social potentials and use? What are some ways that we can test out captions?

Speculative Captioning--Briefly

Some models can be seen in the captions in Night Watch (see Sean's discussion of Night Watch here) and Sherlock (the new series). You can see one example of alternative or speculative captions by Sean in a brief modified clip from Avatar. As Sean points out, the idea is not to replace traditional captions; rather, one of the purposes is to speculate, to come up with ideas about ways that captions could be presented or used, and then to test them out. 

Another influence for me is Walida Imarisha's article about the role of speculative fiction within her larger social justice and prison abolition work. While Imarisha's piece focuses largely on Octavia Butler and prison abolition, one of her opening points is vital:

We [the Left] often forget to envision what could be. We forget to mine the past for solutions that show us how we can exist in other forms in the future. 

That is why I believe our justice movements desperately need science fiction. 

Just as speculative fiction and science fiction can help us see alternate futures, to see past hegemony and imagine alternatives, speculative captioning might help us re-see how we caption, what we caption, what perspectives shape captions, and much more. Frankly, the idea was voiced less than 36 hours ago, so I'm sure there's some evolution coming. I'm not sure if Sean is thinking of SC in way I am. Pretty sure he's going to develop and share what he sees/thinks about SC too.  What I do know, though, is that I'm now seeing captions as much larger than just text on the screen. So much more is there and is possible--I can't undo how I'm thinking of captions now.

Back to the Dissertation

Having all these ideas and thoughts about speculative captioning had me in a good place--a space open to multiple ways that captioning might be able to be used. As I read Heatwole's dissertation, walking through her multiple analyses on identity, fertility, power, and control, I suddenly started asking myself:

  • What if Helena was captioning the video? Would Sarah be "Sestra," Kira as Kira, and Alison and Cosima be "things"--at least until later episodes?
  • If Dyad were running the captions, as a global corporation is likely to do, would the clones be listed as Subject 1, Subject 2, and so forth?
  • What if the captions were only created after the series had ended and was created by a fan who could include short and relevant commentaries?
  • Do these alternative captions represent alternative narratives, or are they simple different angles on the same master narrative generated by the writer and producer?
  • Could these caption tracks be additional, optional tracks available for viewers to watch after they had seen the original, "master" version?
  • In other words, is it possible--or even useful--to remove the voice of objective authority within the captioned narrative and shift that over to a character's voice?
  • What kinds of impacts would such adjustments have on D/deaf viewers and Hearing viewers? How might these impacts differ?
  • What if the alternative caption tracks, i.e. the ones embodying characters' perspectives--like Rachel or Helena--provided additional, supplemental information that expanded and built out the narrative from the traditional captions? Might this be one way to encourage broader use of captions among Hearing folks: supplemental content and material?
  • Could some of these tracks use alternative methods? For example, since Helena clearly sees the world in a very different way--religious, hyper-violent at times, and driven--maybe her captions should be animated and swirly, like inNight Watch, whereas Rachel's captions would be traditional, formal, and proper: embodying the aesthetics of corporate control. Given Alison's character and bubbly, babbling personality, her captions might appear as the scroll up captions so often used in live events [I hate that format, but perhaps the aesthetics express part of her personality?]

End point here: reading scholarship about the primary text (OB) helped me see it in new ways. Just like speculative fiction can help us see through the artificialities of our culture and identify the components which matter, just as speculative captioning will, hopefully, help us envision new ways to caption, feminist and continental theories can help us re-see texts in new ways and thereby crack windows and doors for new perspectives and, most importantly, new modes of action and creation.



Optimal Caption Placement: Ouzts, Snell, Maini, Duchowski

Excited I was to find this conference paper! "Yes," I thought! This will be fascinating. And then, after I finished reading all two pages, I felt disappointed. Is this the authors' fault? A bit. Is it my fault? A bit. You see, I sadly lack the necessary statistical literacy, or my statistical chops are seriously gummed up, to fully make sense of the results section. So, there's that.

Their conclusion, however, reads thus:

"An eye tracking study was presented in which several different captioning styles were examined. Significant differences were found between eye movement metrics depending on the captioning style used, suggesting that captioning styles play an important role in viewing strategies. Participants underwent large amounts of saccadic crossovers and spent much less time reading the captions when captions changed position frequently. Future work is needed to fully examine the implications of these differences" (emphasis added, p. 190).

This makes quite a bit of sense, especially when you consider that they tried four approaches to presenting the captions. (Read the article, heh!) Most notably they tried the traditional captioning positions as well as placing captions above the speakers when present on screen. If not on screen, the captions would be at the bottom. This left me wondering a couple things.

While it may be useful for comprehension to avoid lots of extra or overlapping eye movement, might it not be possible to have the caption placement near speakers occur during intense dialogue and conversation and then shift to traditional (at the bottom) placement when conversation off-screen alternates with significant NSIs (non-speech information)? That might be an interesting approach to captions to test out--especially in dialogue heavy video or film.

As to the article, I am grateful that the authors conducted and shared the research. I just wish the findings would have been more explicitly stated. Then again, there might not have been enough information, or data, to support broader generalizations or suggestions for practice. I respect that. However, in the interest of testing out other approaches to captioning, it would be nice to have some research-driven data from which to launch.

Bringing in the Connotative in #Captions: Nicole Snell's Dissertation on Captions

That title can't sum up Nicole Snell's dissertation, but it does emphasize one of the key points she makes in her work. I'm about twenty pages in and quite enjoying it. It's a dissertation, not an academic article, so that make the reading flow differently; however, the content and focus are quite refreshing.

Particularly love this part:

"This goal [of the dissertation] is undergirded by the hypothesis: since closed captioning changes the passive viewing experience into an active reading one, it can be predicted that users of closed captioning construct connotative and emotional meaning through viewing and meaning making strategies that are different than the strategies individuals who have access to the audio soundtrack and scene action do" (p. 12).

Over the past couple days, I have been trying to think and better understand why I like captions. I seem to have some kind of fixation or attachment to them--something akin to my former obsession with punk rock or specific bands when I was sixteen or eighteen. It's like an itch or fixation or something. Snell's quote, though, helps me better understand my focus on captions though--if only a little bit. Watching the telly, well, just does not do it for me. Boring. I want the captions, and the captions serve and work as a kind of validator of what I'm seeing and hearing. 

Captions also provide additional information. Sometimes it's song lyrics--not always easy to discern if you are just listening; other times, it's background muttering by another character--not always clear in the actual spoken dialogue. If memory serves me right, this happens a fair amount in Orphan Black and similar conspiracy-esque series. Back to Snell's point, though: I'm not just watching, I am also reading.

When I read captions, though, it's not a passive activity. I read and see if all the dialogue is there. I look for specific sounds. I wonder about the presentation of accents--or not. I wonder what non-speech information (NSI) [using a descriptor developed, as far as I know, by Sean Zdenek] is presented and why. I am entertained by the narrative but engage with one form of its representation.

I know that this post's focus is on what and why I like captions--at least part of the reason. The affective is another emphasis Snell cover's in her diss--not just the connotative. If you want more details, please check out her work. Really. This is some exciting thinking!

And yes, this is a processing and working through post.