To run wordpress on Caddy, you need a Caddyfile like this:

www.yourdomian.com
tls youremail@youremailhost.com
root * /usr/local/www/wordpress

php_fastcgi

# Prevent malicious PHP uploads from running
@uploads {
path_regexp path /uploads\/(.*)\.php
}
rewrite @uploads /

@wp-admin {
path not ^\/wp-admin/*
}
rewrite @wp-admin {path}/index.php?{query}
file_server

Then you need to set your permissions right.

sudo chown -R www-data:www-data

as a baseline.