As Apple suggests, you'd better use the Time Machine drive, the Time Capsule in your case, only for Time Machine backup, not for storage of other things.
It seems that you won't lose those RAW images copied to the Time Capsule. Although the Preview can't open the image directly from the Time Capsule, it works when you move it to the Mac desktop. Therefore, your RAW images should be intact. To avoid potential data loss, you should copy the RAW images from the Time Capsule to another external hard drive.
To answer your question: "Is it a good way to back up my photos to the iCloud photo library," you should make it clear that iCloud photo library is not a backup service, it syncs the photos across your Apple devices with the same Apple ID logged in and iCloud Photos enabled on. It is recommended to back up your important files to a physical driver and to the cloud storage so that you can recover your data when one way fails.