Case Studies:
Membership Organizations

The American Society of Plastic Surgeons:
Using Content Analysis to Demonstrate Efficacy and Formulate PR Objectives

by the American Society of Plastic Surgeons & KDPaine & Partners

• ASPS: Nancy E. Ryan, Director of Public Relations & Brian Hugins, Media Relations Associate

• KDPP: Katie Delahaye Paine, President/CEO & Jeremy Willis, Vice President

Background

The American Society of Plastic Surgeons (ASPS) is the world's largest plastic surgery specialty organization. The society advocates for patient safety, supports education and research, and provides information and resources to consumers who are considering plastic surgery.

In 2006, ASPS tasked KDPaine & Partners to continue its longitudinal analysis of media coverage of the organization and its peers in order to help the public relations department and the Executive Committee better understand the efficacy of its PR programming in communicating with the public.

Objectives

• To provide ASPS with valid quantitative information about its media coverage within the plastic surgery sector in a cost-effective manner.

• To facilitate use of said information for the measurement of departmental objectives and in presentations by the Executive Committee to members.

Research Strategies & Tactics

• Collect a census of coverage of ASPS and its peer organizations. A census of a population of 1,571 available sources was more meaningful than a sampling procedure due to the organization's volume of coverage and the importance of targeted placements (see Exhibit D).

  • Tactic: Leverage ASPS' LexisNexis and Cision accounts to cost-effectively collect content.

  • Tactic: Collect content from non-traditional sources, like online health resources (e.g. WebMD) and television entertainment (e.g. Discovery Health Channel programming).

  • Tactic: Use search terms consistent with the 2005 population definition to protect longitudinal integrity.

• Continue using a descriptive content analysis methodology. A descriptive approach was selected above inferential or predictive approaches to keep the investigation on budget and to ensure empirical findings.

  • Tactic: Recruit a member of the public who had considered plastic surgery for health or medical reasons and train him/her to code content in accordance with the coding instrument.

  • Tactic: Analyze all audio and video content as text.

• Construct metrics that are relevant to ASPS's 2006 objectives, without jeopardizing longitudinal trending with data collected in 2005. Revisions to the analysis instrument to keep measures relevant to practitioners in 2006 were made without sacrificing the longitudinal integrity of the data.

  • Tactic: Add subjects as they emerge in order to identify timely media trends.

  • Tactic: Continue to emphasize share of ink, share of discussion and share of coverage that communicated one or more positive message.

  • Tactic: Continue to segment data by ASPS' audiences (Broadcast, Business , General/News, Medical, National, Web and Women's).

• Tailor a report format and schedule that is meaningful for practitioners and management.

  • Tactic: Deliver a quarterly report on the 15th of the month following the analysis period, in time to be on the agenda for the Executive Committee's quarterly conference call.

  • Tactic: Design and deliver a separate executive report that highlights key metrics (share of ink over time, share of discussion over time, share of highly visible coverage over time, share of coverage on key issues, spokesperson message communication and share of audience).

  • Tactic: Keep the full quarterly report as brief as possible (9 pages), so it can be read quickly and fully by all members of the public relations department.

• Provide 24/7 access to a dashboard that facilitates ad-hoc reporting of up-to-date information. A Web dashboard with customizable charts, full drill-down capabilities and compatibility with Microsoft Excel was a cost-effective solution for providing up-to-date information between quarterly reports.

  • Tactic: Represent all of ASPS' key metrics in online charts, with data available immediately after quality checks are completed on the coder's work.

  • Tactic: Train the ASPS public relations staff on using online dashboards for data reporting and investigation.

  • Tactic: Build a dashboard that can be downloaded to Excel with a user-defined analysis period.

Execution

On a daily basis, ASPS delivered qualified and unqualified coverage to KDPP via e-mail for analysis (qualified articles) or storage (unqualified articles). A coder who had been recruited in 2005 to analyze ASPS coverage was retained and trained on the coding instrument and the new subjects that had already emerged in the data. An example of such an emergent addition was the addition of face transplant to the group of tracked, non-exclusive subjects, which was added during the widespread news coverage of the world's first face transplant in late Q4 2005 and into Q1 2006. Identified later in the investigation were medi-spas, volunteer work and tissue engineering, all of which were included as subjects. All other variables remained valid in their a priori conceptualizations.

Once coding had begun, KDPP introduced ASPS' public relations staff members to the Web dashboard interface and its operations. KDPP continued to provide technical support for ASPS staff members, especially as users became more accustomed to customized querying.In addition to quantitative trends in coverage volume and exposure (opportunities to see) for ASPS and its peer organizations, reporting included, but was not limited to, the message communication of members of the ASPS Spokesperson Communication Network, coverage during media attention on the FDA's approval of silicone breast implants, ASPS' lawnmower safety initiative, the frequency of coverage in top publications, and citation of studies from Plastic and Reconstructive Surgery Journal.

Reporting also grew to include several presentations of the message salience of ASPS' press when needed.While reports were restricted in length (3 page executive summary, 9 page full report), analysts focused on both quantitative trends in key metrics and detailed, qualitative examples of highly visible stories, noteworthy coverage and drivers of messaging, in order to draw the connection between outreach efforts and performance.

Results

An independent coder analyzed 1,512 records from January 1, 2006 to December 31, 2006. All records were analyzed for publication, reporters, audience reach, spokesperson communication, article type, prominence, tone, communication of any of nine key messages and discussion of any of 23 subjects.

All four quarterly and four executive reports were delivered on time to ASPS. In addition, the department submitted several data memorandums to its Executive Committee (e.g. Exhibit I). Data from the program was discussed in each of the Executive Committee's quarterly conference calls, and was included in a Presidential presentation to members in July 2006 to demonstrate the department's success in positioning ASPS as a leading source of information about plastic surgery.

Information from this program was used to help formulate the 2007 objectives for the public relations department, to evaluate the performance of the department and, specifically, to evaluate outreach related to Plastic and Reconstructive Surgery Journal studies.
 

Five reasons why KDPaine & Partners should be measuring your success in social media or traditional public relations:

1. All our research is custom designed and performed by human experts: Sometimes computer-aided, always human-coded.
   Our team has been measuring and evaluating communications for 25 years.
2. We measure the tough stuff!
   It's not just about media relations anymore. We measure image, public relationships, reputation, outcomes, and social media conversations.
3. We don't just give you numbers -- we give you advice.
   We are your partners in continuously improving your programs.
4. Scaleable research and services to meet any budget.
   Our solutions start at $3,600.
5. Proven science, proven techniques, proven results.
   We adhere to the Institute for Public Relations Guidelines for Measuring Public Relations. We never provide Advertising Value Equivalency or any numbers based on inexact science.

Click here to purchase the newly released
Social Media Measurement White Paper

25 Women That Rock Social Media

8th Annual Summit on Measurement
Portsmouth, NH
October 6-8, 2010