(CBS NEWS) - Hurricane Matthew has weakened slightly as it moves out of Florida and crawls north along the Atlantic coast. Four storm-related deaths have been reported in the U.S. so far.

The National Hurricane Center said Matthew had sustained winds of 110 mph, making it a very powerful Category 2 storm.

At one point, Matthew reached the strongest Category 5 designation, but it has been slowly weakening as it moves along the coast.

LIVE VIDEO: Watch Florida TV coverage of Matthew
LIVE BEACH CAMS: Florida coasts
TRAVEL: Flight tracker
HURRICANE CENTRAL: Track the storm

Hurricane Matthew sideswiped Florida’s Atlantic coast early Friday, toppling trees onto homes and knocking out power to more than 800,000 people but sparing some of the most heavily populated stretches of shoreline the catastrophic blow many had feared.

Authorities warned that the danger was far from over, with hundreds of miles of coastline in Florida, Georgia and South Carolina still under threat of torrential rain and deadly storm surge as the most powerful hurricane to menace the Atlantic Seaboard in over a decade pushed north.

They warned, too, that the storm could easily take a turn inland.

“It still has time to do a direct hit,” Gov. Rick Scott said in the morning. “This is not over. It could be the worst part of this is yet to come.”

Friday afternoon, the Florida governor’s office released a report from various power companies showing more than 1 million households were without electrical power.

North Carolina Gov. Pat McCrory warned Friday afternoon that up to 15 inches of rain is possible along the coast, according to CBS affiliate WILM.

The state could see the worst flooding since Hurricane Floyd in 1999, McCrory said. The immediate concern is “life-threatening rainfall and water,” he said.

Meanwhile, the magnitude of the devastation inflicted by Matthew as it roared through the Caribbean became ever clearer, with officials in Haiti raising the death toll there to more than 800.

In Florida, Matthew was downgraded to a Category 3 hurricane overnight Thursday, and its storm center, or eye, hung just offshore Friday morning as it moved up the coastline, sparing communities the full force of its 120 mph winds.

Still, it got close enough to knock down trees and power lines, damage buildings and flood streets.

Amy DeLoach, who lives in Merritt Island, posted video purportedly showing her power line exploding:

More hurricane coverage:

See every storm path in Florida for the past 100 years
Here's why South Carolina is better prepared for Matthew than it was for Hugo
Airline cancellations soar to 2,700 through Friday
• 'These guys are crazy,' People hit the beach ahead of the storm
Which 'spaghetti model' is best to gauge hurricanes
A link to full coverage of Hurricane Matthew

In historic St. Augustine, Florida, the downtown district was impassable by noon, with a combination of seawater and rainwater. A giant oak limb had fallen in an old cemetery, and power started going out in some neighborhoods as transformers exploded.

On Georgia’s Tybee Island, where most of the 3,000 residents were evacuated, Jeff Dickey had been holding out hope that the storm might shift and spare his home. But as the rain picked up, he decided staying wasn’t worth the risk.

“We kind of tried to wait to see if it will tilt more to the east,” Dickey said. “But it’s go time.”

Rod Smith (R), hugs his wife Karen as they watch the churning surf on October 6, 2016 on Satellite Beach, Florida. Hurricane Matthew is expected to reach the area later this afternoon. (Photo by Mark Wilson/Getty Images)

In areas the storm had already passed, residents and officials began to assess the damage.

Robert Tyler had feared the storm surge would flood his street two blocks from the Cape Canaveral beach. Tree branches fell, he could hear transformers exploding overnight, and the windows seemed as if they were about to blow in, despite the plywood over them.

But in the morning, there wasn’t much water, his home didn’t appear to have damage on first inspection, and his vehicles were unharmed.

“Overnight, it was scary as heck,” Tyler said. “That description of a freight train is pretty accurate.”

As the storm closed in over the past few days, an estimated 2 million people across the Southeast were warned to clear out.

In the end, Matthew largely skirted the Miami-Fort Lauderdale-Palm Beach areas of over 6 million people and hugged closer to the coast farther north, menacing such cities as Vero Beach, Daytona Beach, Cape Canaveral, St. Augustine and Jacksonville. Farther notice, it threatened such historic cities as Savannah, Georgia, and Charleston, South Carolina.

© 2016 CBS Interactive Inc. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed. The Associated Press contributed to this report.

FLIGHT TRACKER: Latest on air travel affected

Karen Lanman and Don Lanman look out at the churning ocean as Hurricane Matthew approaches the area on October 6, 2016 in Jupiter, Florida. (Photo by Joe Raedle/Getty Images)