The video has sparked a backlash among local residents who say it shows a lack of respect for the community.
In a Facebook post, one woman accused the officer of “taking a shot at a mom****” and accused the police of “using the wrong target”.
The post was shared more than 10,000 times by Tuesday morning, with some calling for an immediate resignation.
Police Chief Greg Hargrave said in a statement that he “deeply regrets” the actions of the officer in the video and was “not comfortable with the way he acted”.
“This is not a video that reflects the values of our department or the community that we serve,” Mr Hargraves said.
“We are very mindful of the fact that we are sworn to protect and serve the citizens of New Jersey, and I am deeply sorry for any discomfort that this has caused.”
The video has also prompted calls for an investigation into the police chief, with one resident asking why a department member is allowed to shoot a suspect who has a gun.
A New Jersey state legislator said on Twitter that “everybody” needs to “take a breath”.
“We need to start listening to what’s going on,” said Assemblyman Robert DeLeo.
“I hope that if I’m elected to office, I will be able to work with the police department to take action to address the issue and stop this from happening in the future,” Mr DeLeos told ABC New York.
The police chief has not responded to requests for comment.
On Tuesday afternoon, the chief tweeted a statement saying he would “not tolerate” such behavior.
Chief Greg Harge says he has no tolerance for officers shooting people on the street, even when they are armed.
He has also ordered an investigation.
Follow Ben Jacobs on Twitter at: @bencjacobs