In the last couple days i found that my sitemap.xml is not automatically update when i publish a new post. As far as i know, sitemap.xml will automatically build using wp-cron.php. And after digging a little bit, i found that something block the wp-cron.php to run. This is the error log shows:
[error] [client xxx.xxx.xxx.xxx] client denied by server configuration: /home/username/public_html/wp-cron.php [error] [client xxx.xxx.xxx.xxx] client denied by server configuration: /home/username/public_html/index.php #xxx.xxx.xxx.xxx: my own ip address
So the error log says that “client denied by server configuration“. This errors may cause by many possibilities, but in my case i misblock the WordPress agent in my htaccess, see my previous post Block Unwanted Visitors To Your Website With .htaccess.
There was a bot scraping my website without honor the robots.txt, the bot use “WordPress 1.01″ user agent. So i block it with ‘*’ (wordpress*) which mean block all user agent which start from ‘wordpress’ word. And this cause misblock the WordPress cron itself. So after i delete that line, all come back to normal again.
So if you are having this issue, be sure to check your .htaccess file, file & directory permission and your apache configuration.
