Case Studies:
Business to Business

Developing Raytheon's Dashboard
Measuring PR's impact when 99.99% of the world can't buy your product.

by Katie Delahaye Paine (This article is condensed from a Golden Ruler of Measurement Award entry.)

The Challenge

For a company like Raytheon, which has as customers only a few hundred people on the planet who can legally purchase its product, normal rules of marketing and promotion don't apply. The notion that you can tie sales of a $1 billion weapon back to a press release is obviously not realistic. Nonetheless, that was our job, and the PR Dashboard team, comprised of PR manager James Fetig and several outside research companies, including KDPaine & Partners and Cymfony, willing took on the challenge.

Background

As in many corporations these days, Raytheon's top management has recently implemented a dashboard system. Each functional area and operating unit has a specific set of gauges by which they measure their success. Ultimately, each unit and functional area feeds its data into an overarching corporate dashboard that enables the CEO Bill Swanson to know whether his company is on track.

While dashboards have been fairly common for Raytheon in areas like manufacturing or finance, incorporating communications into the dashboard system was a whole new challenge. Corporate marketing and communications had already defined specific reputation metrics as their measures of success. Those attitudinal measures, combined with sales metrics such as percent of contract wins, and performance against sales quotas became part of the communications dashboard. The job of the PR Dashboard team was to help add PR into the dashboard system: How would it fit in?

The Objectives

The first step was to meet with the individual responsible for designing and maintaining the gauges that were driving the company. We asked about how he believed PR could tie in to those gauges. His response made our jobs a great deal easier: "We want to see that PR is contributing to the communications effort, that it is helping position the company as we want it to be positioned, and that it is helping us communicate our key messages."

With the objectives clear, all we needed was the research that would give us the data. We crafted the research requirements with the following goals in mind:

1. Competitive analysis was mandatory
Raytheon was clearly not trying to position itself in a vacuum. Whatever research was done had to incorporate competitive results, since the defense business is essentially one long series of competitive bids and announcements of wins or losses.

2. Traditional definitions of success would not suffice, results must be tailored to the unique characteristics of the defense business
The research design could not be based on traditional definitions of success. In the defense industry, coverage is at best neutral, but a quote from a customer in an article or a cover story was a huge victory.

3. A new set of benchmarks would be required
Positive coverage of defense industry topics are few and far between, so we needed to define an industry benchmark that would keep Raytheon's accomplishments and/or failures in perspective. For example, since so few people actually have decision making authority over purchasing Raytheon's products, normal calculations such as opportunities to see were completely irrelevant. However, getting their fair share of ink in the top 25 publications was critical.

4. Results needed to be available in real time
Because of the susceptibility of the defense industry to crisis, the measurement tool had to help Fetig and the communications teams stay on top of potential problems and spot the bad news as soon as it happened.

5. Results had to be actionable
Whatever data the dashboard would produce had to help the team make better decisions. Reports needed to not only reflect what happened, but why it happened, what contributed to the success or failure, and how Raytheon might improve in the future.

The Solution

The solution was a multi-tiered system that incorporated survey and attitudinal research with automatic data gathering and analysis, human review of tonality and messaging, and in-depth reporting that explained the results.

The Attitudinal Study

Raytheon already had an annual communications and reputation study in the works, so it was a relatively easy step to add the requisite questions. In addition to standard questions about the company's reputation relative to its competitors, we also incorporated questions about respondents' reading habits to determine the likelihood of respondents seeing an editorial or news story about the company. We also provided the research company with a full set of statements from the Grunig reputation study ("Guidelines for Formative and Evaluative Research in Public Affairs"), and then incorporated several of them into the final survey instrument.

The Metrics

The next step was to determine what people were seeing, hearing or reading that might contribute to any change in awareness, attitudes or perceptions.

We further defined the metrics that would appear on the dashboard:

  • Share of ink
  • Share of high-visibility focal point articles
  • Share of favorable positioning over time
  • Share of unfavorable positioning over time
  • Share of discussion of mission critical areas
  • Share of discussion of major initiatives
  • Share of leadership positioning
  • Share of mission systems integration positioning
  • Positive coverage vs. the benchmark
  • Negative coverage vs. the benchmark
  • Share of spokesperson visibility quotes
  • Share of quotes
  • Additional data provided monthly details on:
  • Top reporters covering Raytheon more or less favorably than the competition
  • Publications most likely to convey key messages
  • Publications covering Raytheon more or less favorably than the competition
  • In addition to the monthly dashboard reports, Raytheon also had 24/7 accesses to the data via its own brand dashboard.

Implementation:
 Expanding usage, one division at a time.

Implementation of the program began in the fall of 2003. Fine tuning of the benchmarks and the metrics continued during the first six months, as the team learned the capabilities of the system and the realities of the defense industry.

Once the corporate report was up and running, individual business units began to sign on. While the corporate report provided the overall structure and foundation, each new business unit had its own specific programs that it needed to track. Different divisions also had different competitors -- or rather, one division's competitor might be another division's customer, so keeping the various brand definitions clear was critical to the process.

Lessons Learned:
Using the data.

The first reports were distributed to a very limited list of communications and marketing staff to ensure that conclusions were accurate and all data was presented in a clear and easily understandable manner. Once the report format was finalized, the PR team began to expand the sphere of distribution. That's when several major insights developed.

The system allowed Raytheon to easily set up mini-analyses to evaluate the company's visibility on major programs. What quickly became obvious was that on those programs where Raytheon was relatively invisible, the likelihood of winning a bid was much lower. Clearly PR was helping Raytheon generate share of mind among the decision makers.

Another lesson was the usefulness of providing regular feedback to the company's spokespeople. After years of maintaining a distinctly low profile, engineers and program managers were now being asked to play major roles in discussions of potential programs, wins and losses and most importantly, promoting the advanced technology that Raytheon offers. By providing fast and clearly understandable feedback to the spokespeople, Raytheon's PR team has managed to turn around the corporate "shyness" that prevailed in previous years.