That could spell trouble for the Austin, Texas-based grocer. Investors have doubts about whether the company can keep up its long track record of steady sales growth, now that it has saturated the upper-middle-class enclaves on the coasts. Its stock price has been pummeled lately, dropping about 30 percent in the past six months.
"They have to expand beyond where they currently are, and that's kind of the concern," said David McGoldrick, a U.S. analyst at the market research firm Euromonitor International. "Are they going to be able to find customers in other parts of the country, where incomes are lower, that are going to be able to spend at Whole Foods?"
Betting on the middle class is risky. As the country continues to split into rich and poor, middle-of-the-road retailers like J.C. Penney Co. and Sears have suffered.
At the same time, retailers that cater to price-conscious customers -- the middle- and lower-income shoppers who have been squeezed by flat wage growth and a flagging economy -- are thriving. Discount grocers like Aldi and Save-a-Lot proliferated during the recovery. Barring a middle-class revival, upper-crust grocers like Whole Foods could have a harder time finding new customers.
"If you want mass consumption, you need a middle class," said Amy Traub, a senior policy analyst at Demos, a think tank focused on boosting low-income Americans.
Not long ago, Whole Foods enjoyed years of same-store sales growth -- an important retail metric measuring the performance of stores open more than a year -- at a relatively high 8 percent. That's because for many years, Whole Foods was more or less alone in offering a wide variety of organic and natural foods.
Shopping at Whole Foods was also part of an "aspirational" lifestyle, something that consumers sought in a more robust economy. For some, the Whole Foods brand has the same cache as Neiman Marcus or other luxury companies, as Michael Pavone, the CEO of food and beverage marketing company Quench, wrote in a 2012 op-ed for Fast Company.
But Whole Foods is becoming a victim of its own success, according to McGoldrick. Thanks in part to the brand's popularity, now everyone wants to eat healthy food. But most customers can't -- or don't want to -- pay a premium to do so. And Walmart, Kroger and other lower-scale grocers are stepping in to offer organic food at affordable prices to meet the demand.
It seems to be paying off: Kroger's stock price recently soared after the company reported first-quarter earnings that beat expectations. The company's CEO projected 12 to 15 percent profit growth over the year. That growth could come at the expense of Whole Foods' push into the middle market.
Whole Foods has responded by lowering prices and elbowing into new territory. Co-CEO John Mackey recently laid out a plan for future growth that involves expanding from about 350 stores to 1,200 in the next several years and cutting prices even further.
"For a long time, Whole Foods had the field to ourselves, pretty much. That was nice, but we don't any longer," Mackey said. "So, we're adapting to the reality of the marketplace, which is increased competition."
As part of its expansion strategy, Whole Foods is moving into malls, according to Todd Sullivan, a general partner at Rand Strategic Partners. But the company is clustering in shopping centers with higher-end retailers, because lower-end malls are struggling. Middle-class shoppers who once frequented the less upscale places are just now getting back on their feet after the recession. Until they do, all but the nicest malls will probably look like wastelands.
"That's going to be the way it is for a while," said Sullivan.
Whole Foods' price cuts, and its offerings of reasonably priced store-brand and bulk products, haven't yet resulted in a stampede of new customers, according to Sean Naughton, an analyst at Piper Jaffray who covers healthy-living companies. Investors are skeptical about whether the stampede is coming any time soon.
"The perception" is that Whole Foods' groceries are "way overpriced," said Naughton.
Most higher-end retailers are dealing with the dynamic of rising inequality and a shrinking middle class by focusing even more on the small niche of shoppers who can afford their products, said Will McKitterick, an analyst at IBISWorld. Whole Foods is taking the opposite tack, forcing it to strike a balance between keeping its aspirational image and expanding its customer base.
"It's a mystery as to whether it will work," McKitterick said.