Nginx Log Rotation

One of my site is running on Nginx server, and the logs (access log and error log) is getting large very quick . I don’t want to turn it off, because sometime I need to check them. Without using log rotation, I think it’s not good for performance because write into a large file (says 1G) could be slow (correct me if you don’t agree), and another big issue is that it takes long time to open a large log file. So I wanted to rotate the logs.

I found that in fact, the log rotation was already enabled in my server, which is configured by “/etc/logrotate.d/nginx”. Open that file I see the following content,

/var/log/nginx/*log {
    daily
    rotate 10
    missingok
    notifempty
    compress
    sharedscripts
    postrotate
        /etc/init.d/nginx reopen_logs
    endscript
}

The reason my site’s log was not being rotated is because when I set up this site, the log path was not under “/var/log/nginx/.”

It’s easy to fix. Open “/etc/nginx/conf.d/mysite.conf”, and edit the paths of access_log and error_log to be under /var/log/nginx/ will fix the problem.

Note: my server is running ‘CentOS release 6.3’ and the nginx version is 1.2.5, so the path info mentioned in the article may not suitable to your system, but you should be able to do the similar settings.

Auto generate report for previous month

I was asked to write the script to auto generate monthly report about previous month’s data. Because the script will be ran by cron job, the last month is arbitrary. I have to figure out what the time period of the last month is based on the time the script runs. There are two methods,

Method 1: PHP to get previous month

$m =  date("m", strtotime("-1 month") ) ;
$y =  date("Y", strtotime("-1 month") ) ;
$lastmonth = $y . "/" . $m;

And use the following query,

$sql = "select * from testimonials
         where year(insert_date) = '" . $y . "' and month(insert_date) = '" . $m . "'";

Method 2: Use MySQL function to get previous month

$sql = "select * from testimonials
         where `date` BETWEEN date_format(NOW() - INTERVAL $int MONTH, '%Y-%m-01')
         AND last_day(NOW() - INTERVAL $int MONTH)";

After that, I would need to set up the cron job to run every first Monday in a month. Someone suggested the following cron settings, but it actually runs from day 1st to day 7th in every month regardless it’s monday or not.

0 7 1-7 * * 1 /user/bin/php /var/user/task/sometask.php

So the real solution is,

0 7 1-7 * * [ "$(date '+\%a')" = "Mon" ] && /user/bin/php /var/user/task/sometask.php

No gcc on ec2 instance

What I picked is Basic 32-bit Amazon Linux AMI 2011.02.1 Beta (AMI Id: ami-6af08e38)
Try to install mysql-python using easy_install, but get the following error,

unable to execute gcc: No such file or directory
error: command ‘gcc’ failed with exit status 1

Finally figure out there is not gcc on the instance I picked. The solution is easy,
yum install gcc

Cheers!

Quick Note 040411

Use smarty function to check if a smarty variable has been set.

if($smarty->get_template_vars('some_var') == 'off')
 //do something
if ($smarty->get_template_vars('some_var') === null)
//mean smarty variable, some_var is not set

use === or !== instead of == or !=

Says, $foo = true; $foo != ‘hello world’ is not true. So you have to do the following,

$foo = true;
if($foo !== 'hello_world'){
  echo $foo;
}

Order by field

Ordering by the order of values in a SQL IN() Clause, which can be done by ‘order by field(id, [ids from first])’.
For example,

$enames = array('a','b','d');

$sql = "SELECT name from employees 
          WHERE name in ('" . implode("','",$enames) . "') 
          ORDER BY FIELD(name, '" .implode("','",$enames). "')";
 

Nice! Ref – http://stackoverflow.com/questions/396748/ordering-by-the-order-of-values-in-a-sql-in-clause

max_execution_time vs default_socket_timeout

The default setting in php.ini,

max_execution_time = 30
default_socket_timeout = 60

For example, your script need to do socket based streams, which takes 45 seconds, will script timeout because the max_execution_time is 30 seconds?

The answer is NO, because max_execution_time is not affected by system calls or stream operation.
Another important note is that default_socket_timeout is counting before socket responds, as long as it gets response, it will wait forever.

406 error to retrieve jquery.cookie.js file

In a project, I need to use jquery.cookie.js file. It’s weird that this file is already uploaded to the server, and I just could not retrieve it, but got 406 Error. The following is the message I got from Firefox:

Not Acceptable
An appropriate representation of the requested resource /wocai/jquery.cookie.js could not be found on this server.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.

Cannot figure out why, but after I changed the file name to jquery_cookie.js, it fixed the problem. Could the server is not happy file the filename?

PHP Memory Error

Got PHP fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 24 bytes) in /usr/share/pear/Zend/Search/Lucene/Field.php on line 98

Which is not necessary happened to Lucene Script, and it could happen to any script requiring a ton of memory to run, for example parsing a large of XML file.

If you see this error, first check your php.ini file. Looking for “memory_limit”, Mine was set to 128M, which is pretty high already. But that script now asks for more (134M+). So increasing memory_limit should fix the problem. You can also increase memory limit by ini_set(‘memory_limit’, ‘200M’);

Note that you should also check your script to see if you can make it more efficient.

PHP Perl Syntax Check

You can do PHP or Per syntax check under Command line as the following,

Check PHP Syntax

php -l filename.php

Too bad, by -l option, you can see the details, but “Errors parsing filename.php”. To see the detail, you can look into the error log, usually it is phpapplication.log under /var/log/httpd/.
Another note is that if display_errors is set to off in your php.ini file, you will not able to see the phrase error even you try to turn on display_errors by ini_set, because ini_set does not get execute at all.

Check Perl Syntax

perl -wc filename.pl

Some important notes of html form

enctype

  1. if you don’t specify the enctype, then it uses “application/x-www-form-urlencoded” by default.
  2. If you cannot figure why your form cannot upload file, first thing you need to check is if you have set enctype to “multipart/form-data”. Yes, “mulipart/form-data” is a required enctype to upload your file.
  3. Seems there is not different between “application/x-www-form-urlencoded” and “plain/text”, although W3School says, in the case of the former, all characters are encoded before sent, and in the case of the latter, spaces are converted to “+” symbols, but no special characters are encoded. And you don’t need to decode the passing data thru either $_POST or $_GET.

passing data thru url

  1. If the form method is “GET”, then you can also pass the data thru url, and in this case remember urlencode your data if you data have some special character like “&”.  you can do it by PHP function urlencode($data); or JavaScript function escape(data).
  2. If the data is very long, like a few thousands char, then you probably cannot pass it thru url or GET method, use POST from instead. Because per Microsoft, “if you are using the GET method, you are limited to a maximum of 2,048 characters, minus the number of characters in the actual path.”, which is for IE, but I think there are similar limitation in other type of browsers.

input or textarea

For text type data, you can pass it thru <input>, like <input type=”text” value=”<?php echo $somevalue?>” />. In this case, note that,

  1. $somevalue must be escaped if it has double quote in there, otherwise it will mess up the form data. In PHP, you can do it by urlencode() function. In this case, when your retrieve data from $_GET or $_POST, you would not need to decode it, otherwise, there is encoded character, like %22, (stands for double quote)
  2. There is a limit of the length of text type input, not sure what it is, but I got problem in my test when there are 12388 chars. So if you data is too big, then you may just use Textarea, and you don’t need to encode the data in Textarea.