Profiling Perl

Often we write perl scripts that loop a particular action. Not so often we write a perl script that is called in a loop by another tool. My script was attempting to do a http web login to emulate user traffic. However, I was not able to start more than 8 of these scripts a second.

Timing the script inside of perl was actually almost useless. Often you have a profiling marker like:

use Time::HiRes::time;
my $start = time;
# stuff
print "delta: ".(time - $start)."\n";

And that was just telling me I spent 4ms doing my stuff. And so where the heck did the other 108ms occur?

Here was another trick I started using to estimate latency: passing in the start time from the command line:

$ ./script.pl --invoked `date +%s.%4N`

and I would parse it in the script:
my $start;

GetOptions(
   "invoked=f" => \$start,
);
# stuff
print "delta: ".(time - $start)."\n";

Ah…there was my 114ms latency again. So startup of the script must be eating up ~100ms. Well, what could I remove? Anything I did not need to use obviously. Let’s comment out my strictitudinal use statements:

#use strict;
#use warnings;
#use diagnostics;
#use Carp;
#$SIG{ __DIE__ } = sub { Carp::confess( @_ ) };

Wow, I just gained 40ms! But can’t I do better than that? Ah hah, time to use a profiler. After a bit of research, I installed the NYTProf package.

I found some surprising slowdown spots:

  • Time::HiRes – took up to 10ms to load, using `date` takes about 300 microseconds
  • GetOpt::Long – pretty long load time, with a lot of dependencies, what I have now to parse ARGV is not as flexible, but it takes no time to start up.
  • use constant – took up to 45ms to load, about 20ms of that was loading utf8_heavy.ph. Apparently it takes a lot of work to allow any utf8 character to be a legal script constant? Didn’t need it.
  • including ‘use strict’ — in all my own packages, removed them when I wasn’t debugging

Removing GetOpt::Long usage was pretty time consuming. I solved it in about 100 lines of code while still keeping something very close to the original GetOpt syntax, but turning it into a map. Kinda fun, but a bit nail-biting at the same time. Resulting function weighted in at under 100 microseconds.

Result? 6.6 milliseconds run time! I was able to operate the script upto to 68 times a second.

Advertisements

Parenting and Teaching Programming

At what level of programming are we actually imparting the philosophy of human thought? Or, another way, a basic program that prints out a few numbers is truly trivially basic: it merely teaches some programming syntax. However, contrast your smartphone apps to your old DOS/Windows apps. A significan understanding of human haptics and intuition have been codified into the most basic of smart phone apps where your DOS applications might still be more closely related to punch-cards.

Episode 27 of Bad Voltage takes a fascinating turn in the middle of the episode where these parents stop discussing technology in terms of quantifiable capacities and start talking about the philisophical benefits to young programmers. Jono Bacon quotes a collegue:

Programming is smulating how we thing. Teaching programming is teaching a meta-level of how we [make decisions].

And then Bryan Lunduke chimes in with Scratch, a simple programming environment suitable for younger children.

Please teach your kids, especially your girls, how to code. I firmly believe that this is not just a job skill, but an introduction to logic and analysis that is more tangible than any calculus tought, ever. And don’t brag to me about how you used an interval to compute the average runoff from a forested watershed, you’re showing off. You thouch your phone 120 times a day and you’re entirely dependent on it. If you can boil water and put peanut butter on bread, you can program, so stop raising your hands about how difficult it is. Our children will lead lives where software will decide their salaries, their health insurance, their academic opportunities and how they are promoted and ultimately how they promote themselves through life.

Please listen to that episode and tell me why you are not introducing your kids to computer programming.

IntelliJ configuration notes on using HugePages

Java has supported Linux Hugepages for a while, but it is not on by default. I like to squeeze the most performance out of my IDE so I turned it on. And here are the settings I used for Intellij now:

> cat idea64.vmoptions
-XX:+UseLargePages
-XX:LargePageSizeInBytes=2m
-Xms256m
-Xmx768m
-server
-XX:+UseG1GC
-XX:MaxGCPauseMillis=15
-ea
-Dsun.io.useCanonCaches=false
-Djava.net.preferIPv4Stack=true
-Djsse.enableSNIExtension=false
-Dawt.useSystemAAFontSettings=lcd

Forget Foreign Languages and Music. Teach Our Kids to Code | Wired Opinion | Wired.com

Young minds understand coding and logical deduction easily.

The fact that young children can manage such elaborate tasks should be no great surprise, given what we know about their knack for acquiring languages. Five-year-olds trump their elders at learning Spanish or Mandarin because young brains are better (so the theory goes) at formulating “procedural” memories—that is, memories that become so deeply embedded in a person’s psyche that recalling them is a natural reflex rather than a conscious task.

Forget Foreign Languages and Music. Teach Our Kids to Code | Wired Opinion | Wired.com.

However, I don’t like the title of the article. I don’t want to reduce the importance of foreign languages or music. There should be exposure and mentoring in all of these areas for our children.

I had no idea t…

I had no idea that non-Intel processors had boundary restricted byte operations. This certainly could mean general network speedup.

It ultimately turned out that certain device drivers were exposing a problem in the alignment handler. By accessing misaligned IP header fragments, the driver concerned was triggering an alignment exception within an atomic-critical section of kernel code, which was then resulting in the scheduler being called from within the alignment handler. Although the driver was later fixed to improve performance (by using only aligned data), the problem with the alignment handler itself did require fixing to prevent unwanted system crashes.

via The Kernel Column with Jon Masters – Linux Kernel 3.7 | Linux User.

Teaching Kids Programming (DNR, MSDN Blogs)

Second grade is not too early to teach girls or boys to program. This episode of DotNetRocks interviews volunteers who are working with their local schools to introduce elementary programming into after-school clubs and even into school curricula.  The blog post below is an inspiring account of girls learning to program.

Teaching Kids Programming – Teaching Girls to code in Danish – Contagious Curiosity – Site Home – MSDN Blogs.

Programming Standing Up

After listening to a Science Friday report on the correlation between TV time and mortality, I’ve decided to stop sitting at my desk. I’ve propped up my keyboard and monitors with boxes. I’ve done it for a week now. My calves and shins are tired, but my legs don’t get fidgety, or fall asleep or get uncomfortable from sitting for long stretches.

I’m going to have to raise my monitors higher. I think my neck is going to tire out. When I cycle home after work, I definitely notice my neck more–bending my head up. My monitors are lower than they used to be. Happily, I think this indicates that the ergonomics of my monitors were very correct for my sitting posture, because I never noticed any neck strain before.

I’d thought about doing this a year ago, but I immediately jumped into a “what do I have to build” and started over-engineering the issue. Cardboard boxes are working for the present.