I had a new Goodman furnace installed. The installed removed the aluminum exhaust pipe from old furnace that went up to the roof and installed a pvc exhaust for carbon monoxide that exits the house through a side wall.

When the heater kicks on, the exhaust sound from the 2 1/2" PVC pipe is quite loud and can be heard from street.

Why can't the new furnace exhaust go through the original metal exhaust vent through roof?

What can be done to ensure exhaust integrity, but mute the sound?

Also, is there something more attractive, besides the 2 1/2" pvc that sticks out about 10" from side of house (at eye level), that could be used for the exhaust aesthetics?

The pipe they used is required by the furnace suppler. Exhaust from the higher efficiency furnace such as yours creates a lot of moisture that is corrosive to anything other then plastic. So that is the reason for the PVC instead of the metal.
AS far as the noise, Most power vent furnaces do make some noise, I would assume that the pipe is exiting on the side of the home? If this is indeed the case the noise may be amplified if the houses are close together. Not anything you can do about this. You could plant some shrubs, but not to close to help deaden the sound a bit, but it will be a bit noisy anyway.

Paint it to match the home.
If the pipe is the exhaust pipe (air comes out when furnace is running) then it to can be shortened a bit and a Tee fitting installed with the tee looking up and down. But do not cut less then letting six inches stick out. Also be sure that the pipe is not near any windows or vent openings. The install instructions which they should provide you will have the requirements from distances to windows, doors and openings and options for vent terminations.
I would also assume that they pulled permits when they installed this device? If not be sure to get it checked by the local enforcement agency that does this in your area.

one other concern I would have. If you still have a hot water heater installed and its connected to the chimney, You now have what we call an orphaned hot water heater. Meaning the flue gasses that the hot water heater generates may not be enough to create a draft necessary up the chimney when it runs. Many cases the chimney requires a liner of smaller size to fit the hot water heater capacity alone. While its not critical right away, but after time the chimney will begin to corrode from the flue gas water vapors that do not make it up the chimney because its sized to fit both the furnace and hot water heater combined. This is something they should have spoke to you about so check with them or your local gas supplier in this regard.