You will feel the wind resistance more than the weight.
I towed a camper similar to yours in size (6'x13') with my Sienna V6 (HP - around 200 and torque around 200 ft/lbs)
My fuel economy dropped to around 15mpg from around 24mpg. Head winds and climbing up Mt Whitney to Whitney portal campground have a detrimental effect on fuel econ.
The danger is not in the pulling, though; it is in the stopping or slowing.
On the plus side, for the most part the wind resistance of the trailer works as a brake when descending hills and unless you are descending Mt. Whitney the lack of brakes on your trailer should not be a problem. In my case even when locked in first gear I was forced to periodically pull off to let my brakes cool descending Whitney. Brakes can only take so much heat before they begin to feel like ice.
As was said before "easy does it"...