Python

Python remove dictionary key if value is empty

11 Mar , 2015  

Here is a quick snippet if we want to remove dictionary key if value is empty:

1
new_dict = {k: v for k, v in _data.items() if v if v is not ''}

Javascript

Solve issue Fullcalendar count two day as one day

9 Mar , 2015  

This is not bug. When we try to add day in March with start 10-03-2015T10:00:00 and end 11-03-2015T02:00:00, then it will showing one day instead of two day.

To solve this issue, we need to put nextDayThreshold:

1
nextDayThreshold: "00:00"

To make the next day threshold every 00:00

Algorithm

Counting number contain 14 on n-digit number in less than 1 second

8 Mar , 2015  

My fellow Adiyat in our company POLATIC bug me with this question “How to counting number contain 14 in 10 millions numbers less than 1 second?”. Then I thought, it’s easy, let give python a shot!

Famous-characters-Troll-face-Challenge-accepted-140949

1
print len([x for x in range(0, 10000000) if str(x).find('14') > -1])

With result “590040″ it’s takes 9.6s based on SublimeText (yes, i’m kindda hipster guy!). This is not good, no one want to waiting around 10 seconds to get the answer!

Lets break it down by printing sample data for 0-100 and 0-1000

1
2
print [x for x in range(0, 100) if str(x).find('14') > -1]
print [x for x in range(0, 1000) if str(x).find('14') > -1]

And we got :

1
2
3
[14]
[114, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 214, 314, 414, 514, 614, 714, 814, 914]
[Finished in 0.2s]

Wait, i want more, this is not enough to see pattern

1
2
3
4
[14]
[114, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 214, 314, 414, 514, 614, 714, 814, 914]
[1014, 1114, 1140, 1141, 1142, 1143, 1144, 1145, 1146, 1147, 1148, 1149, 1214, 1314, 1400, 1401, 1402, 1403, 1404, 1405, 1406, 1407, 1408, 1409, 1410, 1411, 1412, 1413, 1414, 1415, 1416, 1417, 1418, 1419, 1420, 1421, 1422, 1423, 1424, 1425, 1426, 1427, 1428, 1429, 1430, 1431, 1432, 1433, 1434, 1435, 1436, 1437, 1438, 1439, 1440, 1441, 1442, 1443, 1444, 1445, 1446, 1447, 1448, 1449, 1450, 1451, 1452, 1453, 1454, 1455, 1456, 1457, 1458, 1459, 1460, 1461, 1462, 1463, 1464, 1465, 1466, 1467, 1468, 1469, 1470, 1471, 1472, 1473, 1474, 1475, 1476, 1477, 1478, 1479, 1480, 1481, 1482, 1483, 1484, 1485, 1486, 1487, 1488, 1489, 1490, 1491, 1492, 1493, 1494, 1495, 1496, 1497, 1498, 1499, 1514, 1614, 1714, 1814, 1914, 2014, 2114, 2140, 2141, 2142, 2143, 2144, 2145, 2146, 2147, 2148, 2149, 2214, 2314, 2414, 2514, 2614, 2714, 2814, 2914, 3014, 3114, 3140, 3141, 3142, 3143, 3144, 3145, 3146, 3147, 3148, 3149, 3214, 3314, 3414, 3514, 3614, 3714, 3814, 3914, 4014, 4114, 4140, 4141, 4142, 4143, 4144, 4145, 4146, 4147, 4148, 4149, 4214, 4314, 4414, 4514, 4614, 4714, 4814, 4914, 5014, 5114, 5140, 5141, 5142, 5143, 5144, 5145, 5146, 5147, 5148, 5149, 5214, 5314, 5414, 5514, 5614, 5714, 5814, 5914, 6014, 6114, 6140, 6141, 6142, 6143, 6144, 6145, 6146, 6147, 6148, 6149, 6214, 6314, 6414, 6514, 6614, 6714, 6814, 6914, 7014, 7114, 7140, 7141, 7142, 7143, 7144, 7145, 7146, 7147, 7148, 7149, 7214, 7314, 7414, 7514, 7614, 7714, 7814, 7914, 8014, 8114, 8140, 8141, 8142, 8143, 8144, 8145, 8146, 8147, 8148, 8149, 8214, 8314, 8414, 8514, 8614, 8714, 8814, 8914, 9014, 9114, 9140, 9141, 9142, 9143, 9144, 9145, 9146, 9147, 9148, 9149, 9214, 9314, 9414, 9514, 9614, 9714, 9814, 9914]
[Finished in 0.2s]

Did you see the patterns?

1
2
3
[{1 number}]
[{1 number}, 140..{10 number}, {1 number}, {1 number}, {1 number}, {1 number}, {1 number}, {1 number}, {1 number}, {1 number}]
[{1 number}, {1 number},{10 number}, {3 number} {100 number}, {5 number}, 2014, 2114, {10 number}, 2214, 2314, 2414, 2514, 2614, 2714, 2814, 2914, 3014, 3114, {10 number}, 3214, 3314, 3414, 3514, 3614, 3714, 3814, 3914, 4014, 4114, {10 number},  ...]

Which i can convert into simple list like :

1
2
3
1
9 + 10, ...  => total (19) + 1 = 20
9 + 10, 100, 10 + 10,  10 + 10,  10 + 10,  10 + 10 ... => total (279) + 20 = 299

My assumption, we need (10^n-2) here to capture total number in combination 14, 14x, 14xx, 14xxx … so on.

Second, we see need for recursive function to add result from previous base.

1
f(x) = 10 * f(n-1) + 10^n-2

Let do test with base 2, 3 and 4 on paper :

1
2
3
1
10 + 10
100 + 200

Well, it’s failed on base 3. Let’s think that this formula 10 * f(n-1) missing reduce duplicate number. Let’s reduce it with f(n-2) and recalculate again :

1
2
3
1
10 + 10
100 + 199

Yeay! Let’s write into code :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
"""
Calculate contain number of whatever base 2 number
in given base number
"""
def counting(n):
    if n == 2:
        return 1
    elif n < 2:
        return 0

    return 10 * counting(n-1) - counting(n-2) + 10**(n-2)

if __name__ == "__main__":
    n = 7
    print "The results counting {} base is {}".format(n, counting(n))

And the results is :

1
2
The results counting 7 base is 590040
[Finished in 0.2s]

In Polatic, we always learn and solve enterprise problems with best approach. See pattern and use algorithm to provide quick solution is one of our daily challenge.

Btw, you can check my github in http://github.com/yodiaditya or our company github http://github.com/polatic

Follow this discussion in HackerNews:
https://news.ycombinator.com/item?id=9165357

Django

Django unit test formset

1 Mar , 2015  

When testing page that contains formset in Django using unit test, we may encounter :

1
ValidationError: [u'ManagementForm data is missing or has been tampered with']

This is happend because we POST data without required FORMSET hidden value in form. To solve this issue, inspect element on your formset and insert hidden value in payload data.

Example :

1
2
<input id="id_firstformaset-MIN_NUM_FORMS" name="firstformaset-MIN_NUM_FORMS" type="hidden" value="0">
...

And put this in our payload:

1
2
3
4
5
6
7
8
        self.correct = {
            'name': 'TESTDATA',
            'gender': self.model.FEMALE_STATUS,
            'firstformaset-TOTAL_FORMS': 1,
            'firstformaset-INITIAL_FORMS': 0,
            'firstformaset-MIN_NUM_FORMS': 0,
            'firstformaset-MAX_NUM_FORMS': 1,
        }

implementation

Things to consider when choosing new technology or software in enterprise

27 Feb , 2015  

There are few things that we can use as standar for choosing or adopting new technology in our company.

The List

We gave each team a list of features to look for in their assigned templating solution. The idea was to fill out a score, from one (poor) to five (excellent), for each item:

  1. DRY: how DRY is the the technology? is there support for code-reuse and partials?
  2. i18n: is there support for translations and multiple languages?
  3. Hot reload: are changes visible immediately or is there a compile/deploy cycle?
  4. Performance: how long does it take to render in the browser and server?
  5. Ramp-up time: how is the learning curve?
  6. Ramped-up productivity: once you’ve ramped-up, how fast can you build things?
  7. Server/client support: can support both client-side and server-side?
  8. Community: is there an active community using this project? Can you google issues?
  9. Library agnostic: are there dependencies on other libraries?
  10. Testable: how hard is it to write unit and integration tests?
  11. Debuggable: is it possible to step through the code while it’s running to track down errors?
  12. Editor support: is there an editor with auto-complete, syntax highlighting, error checking, etc?
  13. Maturity: is this a relatively stable project or still experimenting and churning?
  14. Documentation: how is the documentation?
  15. Code documentation: do the software encourage/require documentation/comments?

#inspired by https://engineering.linkedin.com/frontend/client-side-templating-throwdown-mustache-handlebars-dustjs-and-more

Javascript

Example make reusable javascript app for modal progress bootstrap

26 Feb , 2015  

This is example build reusable app in Javascript for modal progress in Boostrap:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
/**
 * Re-usable app for modal box progress
 */
var progressBox;
progressBox = progressBox || (function() {
    var pleaseWaitDiv =  $('<div class="modal hide" id="pleaseWaitDialog" data-backdrop="static" data-keyboard="false"><div class="modal-header"><h1>Processing...</h1></div><div class="modal-body"><div class="progress progress-striped active"><div class="bar" style="width: 100%;"></div></div></div></div>');
    return {
        showPleaseWait: function() {
            pleaseWaitDiv.modal();
        },
        hidePleaseWait: function() {
            pleaseWaitDiv.modal('hide');
        },
    };
})();

To use, we just need to :

1
progressBox.showPleaseWait();

Javascript

How to hide search toolbar in Datatables

25 Feb , 2015  

Here is a quick snippet to hide search toolbar in datatables and working along with Bootstrap

1
2
3
4
var settings = {
            sDom: 'lrt<"col-sm-6"i><"col-sm-6"p>',
            serverSide : true,
...

Javascript

Set bootstrap datetimepicker input value date

25 Feb , 2015  

Here is a quick way to set value bootstrap datetimepicker input with date object:

1
2
3
4
5
6
$("#id_start_date, #id_end_date").datetimepicker({
    format: "DD-MM-YYYY hh:mm"
});

$('#id_start_date').data("DateTimePicker").setDate(start);
$('#id_end_date').data("DateTimePicker").setDate(end);

Javascript

Datatables make link in row records

20 Feb , 2015  

Here is a quick snippet to make record as links in Datatables :

1
2
3
4
5
6
7
8
columns : [
              {
                "mRender": function ( data, type, full ) {
                 // 'full' is the row's data object, and 'data' is this column's data
                 // e.g. 'full[0]' is the comic id, and 'data' is the comic title
                 return '<a href="' + full['uuid'] + '">' + full['created_by']+ '</a>';
               }
];

Server

Change / Reset / Update Key Pem Amazon EC2 Simple

19 Feb , 2015  

Here are simple tricks to change or update key PEM in Amazon EC2. At this example, I’m using Ubuntu.
To reset or change key pem, all you need just follow this simple steps:

Let say, currently we have SERVER_01 and KEYPEM_01.

1
2
3
4
5
6
1. Launch a new instances using "launch more like this" (right-click on instances)
2. Create a new server SERVER_02 and new keypem KEYPEM_02
3. Copy your KEYPEM_01 into SERVER_02 using "rsync"
4. Login into SERVER_02, then "rsync" ".ssh/authorized_keys" to SERVER_01 "/.ssh/authorized_keys"
5. Basically, we just replace the "authorized_keys" from new to old
6. We can login to SERVER_01 using KEYPEM_02

Simple! :D

Django

Django check related model and check field if exists in model

19 Feb , 2015  

We can check if the field is related model in Model by :

1
model._meta.get_field("name_of_field").get_internal_type() == "ForeignKey"

Then we can check if the field name is exists in model by :

1
model._meta.get_field_by_name(_name)

Some example code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
if self.model._meta.get_field(field_name).get_internal_type() == "ForeignKey":
    # get related model
    # http://stackoverflow.com/questions/10347210/django-foreign-key-get-related-model
    related_model = self.model._meta.get_field(field_name).rel.to

    # Checking the default field for related model
    check_name = None
    for _name in ['name', 'code']:
        try:
            related_model._meta.get_field_by_name(_name)
        except FieldDoesNotExist:
            pass
        else:
            check_name = _name
            break

Python

Mapping Ajax Filter and Search Datatables to Django with Rest Framework

16 Feb , 2015  

Datatables sending ajax parameter for filtering and search in very complicated way. Which we need to mapping and parse based on each column.

Some example request :

1
http://localhost:8000/api/somemodel/?format=json&sEcho=4&iColumns=7&sColumns=%2C%2C%2C%2C%2C%2C&iDisplayStart=0&iDisplayLength=10&mDataProp_0=&sSearch_0=&bRegex_0=false&bSearchable_0=true&mDataProp_1=cif&sSearch_1=22222&bRegex_1=false&bSearchable_1=true&mDataProp_2=name&sSearch_2=&bRegex_2=false&bSearchable_2=true&mDataProp_3=gender&sSearch_3=&bRegex_3=false&bSearchable_3=true&mDataProp_4=ktp&sSearch_4=&bRegex_4=false&bSearchable_4=true&mDataProp_5=location&sSearch_5=&bRegex_5=false&bSearchable_5=true&mDataProp_6=&sSearch_6=&bRegex_6=false&bSearchable_6=true&sSearch=&bRegex=false

I have no idea about where the filtering and search query come from in this request. Therefore, we need to auto-mapping it by :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import re
from urlparse import parse_qs

"""Filtering get params from datatables and mapping into django filter"""
def filter_request(self):
    # Filtering input
    input_request = parse_qs(self.request.GET.urlencode())
    query_dict = {}
   
    for key, value in input_request.iteritems():
        if(re.search('sSearch_', key)):
            if value:
                key_index = input_request.get('mDataProp_%s' % key.split('_')[1])

                if key_index is not None:
                    query_dict[key_index[0]] = value[0]

    return query_dict

And we get nice results :

1
{u'some_field': u'some_value', ...}

Uncategorized

Disable sort on first column in Jquery Datatables

15 Feb , 2015  

Dunno if this is bug or not, but disable sort in HTML nor Javascript will not affect for first column sorting in Datatables. The solution just define bSort = false

1
2
3
4
5
6
var table = $('#table-index').dataTable({
            serverSide : true,
            sAjaxSource : url,
            autowidth : false,
            fnServerData : setupAjax,
            bSort: false,

Uncategorized

Solve ScrollTop Jquery not working in Boostrap because overflow:auto

14 Feb , 2015  

Yes, ScrollTop Jquery not working in Boostrap because overflow:auto. To solve this, we need to use div that using “overflow:auto” and solve by :

1
2
3
4
5
6
7
8
9
10
11
12
<script type="text/javascript">
           $(document).ready(function() {
               
                /* smooth scrolling for scroll to top */
                // scroll to top action
                $('a.scroll-top').click(function(event) {
                    jQuery('#main').animate({
                         scrollTop: jQuery('#main').scrollTop() + jQuery('#main').position().top - 3500
                     }, 500);
                });
            });
</script>

Uncategorized

Solve there was problem sending the command to the program excel

2 Feb , 2015  

When open excel files, I got this errors :

1
there was problem sending the command to the program

To solve this issues in Windows :

1. launch Microsoft Excel

2. Go to File -> Options -> Advanced

3. Scroll down and find “Ignore other application that use DDE” (see images below) and tick them

Capture

Problem solved!

Javascript

Jquery Datatables modify fnserverData and pagination

28 Jan , 2015  

Here is a quick snippet for custom AJAX data for Jquery datatables :

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
 var table = $('#table-index').dataTable({
                "serverSide": true,
                "searching": false,
                "ordering": false,
                "sAjaxSource": url,
                "bStateSave": true,
                "fnStateSave": function (oSettings, oData) {
                    localStorage.setItem( 'DataTables', JSON.stringify(oData) );
                },
                "fnStateLoad": function (oSettings) {
                    return JSON.parse( localStorage.getItem('DataTables') );
                },
                "fnServerData": function ( sSource, aoData, fnCallback, oSettings ) {
                 oSettings.jqXHR = $.ajax( {
                   "url": sSource,
                   "data": {'page': Math.ceil(oSettings._iDisplayStart / oSettings._iDisplayLength) + 1},
                   "success": fnCallback,
                   "error": function (e) {
                       console.log(e.message);
                   }
                 });
               },
                "columns": [
                    { "data": "name" },
                    { "data": "gender" },
                    { "data": "location" },
                    { data: null,
                      className: "center",
                      defaultContent: '<a href="#" class="edit_data"><i class="fa fa-pencil"></i> Edit</a> |  <a href="" class="remove_data"><i class="fa fa-trash"></i> Remove</a>'
                    }
                ],
            });

For pagination, the key is :

1
 "data": {'page': Math.ceil(oSettings._iDisplayStart / oSettings._iDisplayLength) + 1},

Python

Using format instead of %s for string replacement

23 Jan , 2015  

Usually we use “%s” for string replacement in Python:

1
"asddas %s" % ('hello',)

There is better way to do it:

1
2
3
"hello {a} {b} {c}".format(a='error', b='hello', c='world')
"hello {0} {1} {2}".format("hello", "world", "fine")
"hello {}".format("hello")

Uncategorized

Decoder jpeg not available in Ubuntu 14.04

15 Jan , 2015  

To solve this issue :

1
sudo apt-get install libjpeg-dev libfreetype6-dev zlib1g-dev libpng-dev xfonts-base xfonts-75dpi libjpeg8-dev libjpeg-turbo8-dev

Django

Your branch and ‘origin/master’ have diverged, and have 1 and 1 different commit each, respectively.

15 Jan , 2015  

I got this problem:

1
2
3
4
5
6
λ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 1 and 1 different commit each, respectively.
  (use "git pull" to merge the remote branch into yours)
nothing to commit, working directory clean

Solution

1
2
λ git reset --hard origin/master
HEAD is now at 63813ef fix localdb

, ,

Django

Django 1.7 reset migration solved

12 Jan , 2015  

Here is a quickstep to reset database migration in Django 1.7 :

1
2
python manage.py makemigrations --empty your_app
python manage.py makemigrations your_app

Django

Git rollback last commit in remote

12 Jan , 2015  

To rollback last commit :

1
2
git reset HEAD~1 --hard
git push -f

Django

Git checkout track from remote branch

12 Jan , 2015  

Let’s say somebody already push their branch in remote called “finances”. We want to fetch and working on this branch from our local. So, the solution :

1
2
3
4
5
git fetch --all

Fetching origin
From github.com:polatic/vasham-erp
 * [new branch]      finance    -> origin/finance

Then, we checkout with tracking

1
2
3
git checkout -t origin/finance
Branch finance set up to track remote branch finance from origin.
Switched to a new branch 'finance'

Now ready for working :

1
git checkout finance

Django

Accelerate django development server with django-devserver

9 Jan , 2015  

As we know what django development server running on single thread. This is make development a bit slow. There is some alternative, by using django-devserver

Here is the quickstep:

1. Install the app

1
pip install git+git://github.com/nealtodd/django-devserver

2. Edit your local_settings.py

1
2
3
4
5
INSTALLED_APPS =  ('devserver',) + INSTALLED_APPS

MIDDLEWARE_CLASSES = MIDDLEWARE_CLASSES + (
   'devserver.middleware.DevServerMiddleware',
)

3. Then run

1
python manage.py runserver

Django

Ubuntu Check Memory Usage by Process

9 Jan , 2015  

Here is a quick snippet for check memory usage by process :

1
ps aux | sort -rn -k 5,6 | less

Django

Solve Django Filter queryset result duplicates with Pagination

5 Jan , 2015  

When we filter using multiple filter and mixed with django pagination, somehow the results are duplicates. This is incorrect. Apparently, looking at the source code https://github.com/alex/django-filter/blob/master/django_filters/filters.py#L105, I found this line:

1
2
3
4
q = Q()
for v in value:
    q |= Q(**{self.name: v})
return qs.filter(q).distinct()

By default, django-filter not using “AND” condition and that’s why it’s showing duplicates.

To fix this, then we can use distinct when passing object_list in paginator, example:

1
results = paginator(request, PlanningOrder, 'farmseason', object_list.qs.distinct())

Django

Setup Django and Timezone follow server datetime (Ubuntu)

5 Jan , 2015  

Here is quickfix to setup Django working with server timezone.

1. Set Server Timezone

Check by :

1
2
date
Mon Jan  5 10:54:38 WIB 2015

If this not correct or you want to change server timezone, then edit “/etc/timezone”

1
Asia/Jakarta

Then do :

1
2
sudo dpkg-reconfigure --frontend noninteractive tzdata
date

Now you should see the updated server time.

2. Django
We need to edit settings.py add edit this line:

1
2
TIME_ZONE = 'Asia/Jakarta'
USE_TZ = True

Restart your django server (uwsgi or wsgi) and do testing

Django

Django django-admin.py startproject ImportError

27 Dec , 2014  

Suddenly i got this error in Windows when trying to start a new Django project :

1
django-admin.py startproject ImportError: No module named django.core

The solution :

1
python C:\Python27\Scripts\django-admin.py startproject example