Art Cashin, director of floor operations at UBS, said the roughly 1 percent decline in the Dow transports was weighing on equities. Pressured primarily by airlines, the index is on track for its worst week since March 27.
The transports are down 6.8 percent in the last six months, versus the Dow Jones industrial average’s 2.3 percent gain.
The Dow traded about 50 points lower. The S&P 500 and the Nasdaq attempted slight gains, with the latter trying to hold above its closing high of 5,092.09.
Stock index futures turned mildly negative on the inflation report and the 2-year Treasury yield topped 0.61 percent, but longer-end bond yields initially held steady before creeping higher. The 10-year yield traded near 2.22 percent.
The dollar turned positive, gaining nearly 1 percent with the euro lower at $1.10.
The cash bond market will close early today at 2 p.m. ET ahead of the long weekend.
“Investors are starting to price in … a rate hike some time in 2015,” said Andrew W. Ferraro, wealth advisor at Strategic Wealth Partners. The consumer price index came in “a little higher. That may suggest a slightly earlier rate hike.”
Fed fund futures priced in a 45 percent chance of rate hike in September, up from 40 percent earlier and 35 percent last week.
Read MoreDivergence creates spooky undercurrent for stocks
The Labor Department said on Friday its Consumer Price Index (CPI) rose 0.1 percent last month, with the core figure discounting food and energy costs up 0.3 percent, for the largest gain since January 2013.
Economists polled by Reuters had forecast the CPI edging up 0.1 percent from March and dipping 0.1 percent from a year ago.
“I think it’s a decent sign for the economy,” said Todd Hedtke, vice president for investment management at Allianz Investment Management. “I don’t think it’s a good sign for capital markets. It’s been a ‘goldilocks’ market for so long. You’re going to see some pressure. But the bottom line, the inflation inching north is a positive sign. We need to see a little more of that spending.”
Read MorePay up now, but gas is going to get much cheaper
Much of the recent economic data indicated moderate growth, but not enough to warrant an interest rate hike soon.
Despite better jobs data, consumer spending and price increases were mostly muted. The Federal Reserve is watching for enough strength in employment and inflation to support a rate hike.
“I think it wasn’t an outstanding number,” TD Ameritrade chief strategist JJ Kinahan said of the CPI read. “But when you’re at all-time highs and no blow-away numbers it’s an excuse to sell.”
30-day DJIA performance
“A low volume market can be volatile,” said Art Hogan, chief market strategist at Wunderlich Securities. We may get “an outsized move on anything that is perceived to be different (in Yellen’s speech).”
The Dow Jones Industrial Average traded down 22 points, or 0.13 percent, at 18,261, with Boeing leading laggards and American Expressand Apple the greatest advancers.
The S&P 500 traded down 0.41 points, or 0.02 percent, at 2,130, with telecommunications leading seven sectors lower and information technology the greatest advancer.
The Nasdaq traded up 11 points, or 0.22 percent, at 5,101.
About four stocks declined for every three advancers on the New York Stock Exchange, with an exchange volume of 160 million and a composite volume of 731 million in late morning trade.
Crude oil futures for July fell 73 cents to $60.00 a barrel on the New York Mercantile Exchange. Gold futures gained $2.20 to $1,206.30 an ounce as of 11:05 a.m.
Earnings out on Friday included Campbell Soup, Deere, Foot Lockerbefore market open.
Lions Gate Entertainment came in 7 cents above estimates with adjusted quarterly profit of 39 cents per share, but revenue fell considerably below analyst forecasts.
BlackBerry will buy back about 2.6 percent of its outstanding shares, to negate potential negative effects of a proposed employee stock purchase plan.
Hewlett-Packard reported adjusted quarterly profit if 87 cents per share, 2 cents above estimates, though revenue was slightly shy of forecasts. The company also issued weaker-than-expected current quarter guidance. Investors are taking note of a positive development—lower-than-expected expenses for the separation of its personal computer and printer businesses into a separate company.
Deere & Co. earned $2.03 per share for its latest quarter, beating estimates of $1.55 despite a slight revenue shortfall. Deere noted a weak global agricultural sector, but said good execution aided its bottom line.
Foot Locker beat estimates by 6 cents with earnings of $1.29 per share. Revenue and same-store sales were above analyst forecasts, and the company said the quarter was the most profitable in its history.
Campbell Soup earned an adjusted 62 cents per share for its latest quarter, 10 cents above estimates, though revenue fell short due to currency effects. Campbell said its full-year sales would fall towards the lower end of its projected range, but earnings would be at the more favorable end.
Read MoreEarly movers: DE, FL, CPB, HPQ, GPS, ROST & more
European equities were mixed on Friday as investors focused on a central bank meeting in Portugal and Yellen’s speech.
Mario Draghi, president of the European Central Bank (ECB), spoke at the event in Sintra, Portugal earlier Friday. He said that the economic outlook for the euro zone is looking “brighter today than it has done for seven long years.”
The S&P 500 ended mildly higher on Thursday above its previous record close of 2,129.20 to post its 10th closing high for the year.
“Investors are going into this weekend knowing there isn’t any catalyst to force the Fed to move anytime soon and that means we’re all data-dependent,” said Adam Sarhan, CEO of Sarhan Capital.
Durable goods orders and another read on first-quarter GDP come out next week.
—Reuters and CNBC’s Patti Domm and Peter Schacknow contributed to this report.