Am I understanding your post correctly? He is only 6 weeks old? If this is correct, have you taken him to a vet yet? He is probably not up to date on shots, he has not had his heart checked which the breeder should do at 12 weeks before they send the puppy home. Which means you got him 6 weeks too early. Who put him on a raw diet? I would talk to a vet before switching. The panting may or may not be normal. Newfs get hot very easily, so if you have him in a crate when you are not supervising him (and I hope you do), remove any bedding for starters. You might put a small stuffed toy in the crate since he was taken from his sibling way to early and may miss their cuddling. He will need extra socialization when he has had the rest of his shots (he is at serious risk for parvo until he has all is vaccines) and especially will need bite inhibition training, which he would have learned from his siblings had he stayed the required time with them. Line up a good rewards based trainer now, before you need one. Ask if you can pick their brain before you start class. Newf owners need formal training instruction from someone skilled in canine behavior...period. It will make life much easier for you and is totally worth the money. Ideally he should start puppy class as early as 13 weeks old, but this virus crap may interfere. If you can talk a trainer into meeting outside or at your or their home, go for it. If you can enroll in a class setting, even better.
Also, if you can find a regional Newf club near you, reach out to them for help. Join if you can; you'll learn a lot and have a lot of doors opened to you. Be sure to search the Newfoundland Club of America website for info, too.
This forum was down for a long while and is just now running again, so don't expect tons of answers, but do check regularly for replies. I don't think the word has gotten out yet that the forum is online again.
99 percent of the long time member's here have very reliable info and are easy to talk to. We don't take kindly to bullying or disrespect, so I hope that will make you feel more comfortable. These are people with lots of experience in all areas of dog ownership and of course masters of Newf wisdom.
Now, I'm certain that your puppy will be fine with proper vet care and training. It is irrelevant at this point where he came from or whether his breeders were irresponsible or not. He is your little (soon to be big) love and he needs to know you will take care of him. If you can put the anxiety of his previous circumstances behind you, and can relax and just have fun with him, you'll soon be rewarded with the most loyal and loving relationship you'll ever have.
Please stay in touch. Ask all the questions you like and hopefully, as the forum gets going again, you'll have more Newf friends that you can imagine.
Best of luck...and what is your baby's name?