python logo

Evaluating Python Implementations

Here, we the measured programs grouped four different code categories that showed distinct execution times among the different implementations. The majority of the code (arithmetic, comparison, bitwise and logical operations with all the built-in types, basic functions, string management, member access, invocation, exception handling...) has been classified in a general common group. The rest of the categories that showed different relative performances are multi-threading, IO and metaprogramming operations.
  1. Microbenchmarks
  2. Benchmarks
  3. Programs
  4. Large Scale Applications

Microbenchmarks

Common characteristics
Number Microbenchmark Test
Integer and Long Arithmetic
1 JavaGrande JGFArithAddInt
2 JavaGrande JGFArithMultInt
3 JavaGrande JGFArithDivInt
4 Pybench SimpleIntegerArithmetic
5 Tommti TommtiIntArithmetic
6 JavaGrande JGFArithAddLong
7 JavaGrande JGFArithMultLong
8 Tommti TommtiLongArithmetic
9 JavaGrande JGFArithDivLong
10 Pybench SimpleLongArithmetic
Floating Point Arithmetic
1 JavaGrande JGFArithAddFloat
2 JavaGrande JGFArithMultFloat
3 JavaGrande JGFArithDivFloat
4 Pybench SimpleFloatArithmetic
5 JavaGrande JGFArithAddDouble
6 JavaGrande JGFArithMultDouble
7 JavaGrande JGFArithDivDouble
8 Tommti TommtiDoubleArithmetic
Rational Arithmetic
1 Custom RationalArithAddInt
2 Custom RationalArithMultInt
3 Custom RationalArithDivInt
Complex Aritmethic
1 Custom AddComplex
2 Custom MulComplex
3 Custom DivComplex
4 Pybench SimpleComplexArithmetic
Bit Arithmetic
1 Custom GGBench
2 Custom LLBench
3 Custom BORBench
4 Custom BAndBench
5 Custom BXorBench
Common Mathematical functions
1 JavaGrande JGFMathAbsInt
2 JavaGrande JGFMathAbsLong
3 JavaGrande JGFMathAbsFloat
4 JavaGrande JGFMathAbsDouble
5 JavaGrande JGFMathMaxInt
6 JavaGrande JGFMathMaxLong
7 JavaGrande JGFMathMaxFloat
8 JavaGrande JGFMathMaxDouble
9 JavaGrande JGFMathMinInt
10 JavaGrande JGFMathMinLong
11 JavaGrande JGFMathMinFloat
12 JavaGrande JGFMathMinDouble
13 JavaGrande JGFMathSqrtDouble
14 JavaGrande JGFMathExpDouble
15 JavaGrande JGFMathLogDouble
16 JavaGrande JGFMathPowDouble
Trigonometrical, rounding and random functions
1 JavaGrande JGFMathSinDouble
2 JavaGrande JGFMathCosDouble
3 JavaGrande JGFMathTanDouble
4 JavaGrande JGFMathAsinDouble
5 JavaGrande JGFMathAcosDouble
6 JavaGrande JGFMathAtanDouble
7 JavaGrande JGFMathAtan2Double
8 Tommti TommtiTrig
9 JavaGrande JGFMathFloorDouble
10 JavaGrande JGFMathCeilDouble
11 JavaGrande JGFMathRintDouble
12 JavaGrande JGFMathRoundFloat
13 JavaGrande JGFMathRoundDouble
14 JavaGrande JGFMathIEEERemainderDouble
15 JavaGrande JGFMathRandom
Comparisons
1 Pybench CompareIntegers
2 Pybench CompareFloats
3 Pybench CompareFloatsIntegers
4 Pybench CompareLongs
5 Custom CompareObjectsReference
6 Custom CompareObjectsByValue
Flow control
1 Pybench IfThenElse
2 JavaGrande LoopFor
3 JavaGrande LoopWhile
4 JavaGrande LoopReverseFor
5 Tommti TommtiNestedLoop
6 Pypy translator test_loop
7 Pypy translator test_loop_other_count
8 Pypy translator test_loop_unrolled
9 Pybench NestedForLoops
10 Pybench SequentialForLoops
Member access
1 Pybench NormalInstanceAttribute
2 Pybench NormalClassAttribute
3 Pybench SpecialClassAttribute
4 Pybench SpecialInstanceAttribute
5 Pypy translator test_count_in_attr
6 Pypy translator test_count_with_True
7 Pypy translator test_count_with_global_increment
8 Custom VariableAccess
9 Pypy translator test_count_in_slot
10 Pypy translator test_count_in_global
11 Pypy translator test_count_increment_in_global
12 Pypy translator test_count_in_global2
13 JavaGrande AssignSameScalarLocal
14 JavaGrande AssignSameScalarInstance
15 JavaGrande AssignSameScalarClass
16 JavaGrande AssignOtherScalarInstance
17 JavaGrande AssignOtherScalarClass
18 JavaGrande AssignSameArrayLocal
19 JavaGrande AssignSameArrayInstance
20 JavaGrande AssignSameArrayClass
21 JavaGrande AssignOtherArrayInstance
22 JavaGrande AssignOtherArrayClass
Invokation
1 Pybench PythonFunctionCalls
2 Pybench ComplexPythonFunctionCalls
3 Pypy translator test_call_function
4 Pypy translator test_call_nested_function
5 Pypy translator test_call_nested_function_other_count
6 Pypy translator test_call_nested_function_many_args
7 Pypy translator test_call_function_with_arguments_in_cellvars
8 Pypy translator test_call_function_without_arguments_in_cellvars
9 Pypy translator test_call_1
10 Pypy translator test_call_2
11 Pypy translator test_call_3
12 Pypy translator test_call_4
13 Pypy translator test_call_default_1
14 Pypy translator test_call_default_2
15 Pypy translator test_call_keyword_1
16 Pypy translator test_call_keyword_2
17 Pypy translator test_call_keyword_3
18 Pybench BuiltinFunctionCalls
19 JavaGrande MethodSameInstance
20 JavaGrande MethodSameClass
21 Pypy translator test_call_method_of_old_style_class
22 Pypy translator test_call_method_of_new_style_class
23 JavaGrande MethodSameSynchronizedInstance
24 JavaGrande MethodSameSynchronizedClass
25 JavaGrande MethodOtherInstance
26 JavaGrande MethodOtherInstanceOfAbstract
27 JavaGrande MethodOtherClass
28 Pybench Recursion
29 Pybench PythonInnerClassMethodCalls
Exception Handling
1 JavaGrande ExceptionThrow
2 JavaGrande ExceptionNew
3 JavaGrande ExceptionMethod
4 Pybench TryRaiseExcept
5 Tommti TommtiExcept
6 Pypy translator test_raise_builtin_exception
7 Pypy translator test_raise_user_exception
8 Pypy translator test_except_specific_builtin_exception
9 Pypy translator test_except_multiple_builtin_exception
10 Pypy translator test_except_specific_user_exception
11 Pypy translator test_except_multiple_user_exception
12 Pypy translator test_reraise
13 Pybench TryExcept
14 Pypy translator test_try_except
15 Pypy translator test_try_except_else
16 Pypy translator test_instantiate_builtin_exception
17 Pypy translator test_instantiate_user_exception
18 Pybench WithFinally
19 Pybench TryFinally
20 Pybench WithRaiseExcept
21 Pypy translator test_try_except_finally
Data creation
1 JavaGrande CreateArrayInt
2 JavaGrande CreateArrayLong
3 JavaGrande CreateArrayFloat
4 JavaGrande CreateArrayObject
5 JavaGrande CreateObjectBase
6 JavaGrande CreateObjectSimple
7 JavaGrande CreateObjectSimpleConstructor
8 JavaGrande CreateObjectSimple1Field
9 JavaGrande CreateObjectSimple2Field
10 JavaGrande CreateObjectSimple4Field
11 JavaGrande CreateObjectSimple4fField
12 JavaGrande CreateObjectSimple4LField
13 JavaGrande CreateObjectSubclass
14 JavaGrande CreateObjectComplex
15 Pybench CreateInstances
16 Pybench CreateNewInstances
17 Pypy translator test_simple_loop_with_old_style_class_creation
18 Pypy translator test_simple_loop_with_new_style_class_creation
19 Pypy translator test_simple_loop_with_new_style_class_new
Strings
1 Pybench StringCreation
2 Custom StringAccess
3 Pybench StringConcat
4 Pybench StringSubstring
5 Custom StringSearch
6 Pybench StringCompare
7 Pybench StringCompareInterned
8 Pybench StringMethods
9 Pybench StringMappings
10 Tommti TommtiStringConcat
11 Pybench StringCreationUnicodeWithConcat
12 Pybench StringConcatUnicode
13 Pybench StringSubstringUnicode
14 Pybench StringCompareUnicode
15 Pybench StringUnicodeMethods
16 Pybench StringUnicodeMappings
17 Pybench StringUnicodeProperties
18 Pypy translator test_find_worstcase
19 Pypy translator test_count_worstcase
20 Pypy translator test_find_pattern16
21 Pypy translator test_find_pattern8
22 Pypy translator test_find_pattern4
23 Pypy translator test_find_pattern2
24 Pypy translator test_find_pattern1
25 Pypy translator test_simple_formatting
26 Pypy translator test_dict_formatting
27 Pypy translator test_number_formatting
28 Pypy translator test_repr_formatting
29 Pypy translator test_format_unicode
30 Pypy translator test_format_long
Module import
1 Pybench SecondImport
2 Pybench SecondPackageImport
3 Pybench SecondSubmoduleImport
Data structures
1 Pybench TupleSlicing
2 Pybench SmallTuples
3 Tommti TommtiHeapsort
4 Tommti TommtiMatrixMultiply
5 Custom SetManagement
6 Pybench ListManagement
7 Pypy translator test_list_append
8 Pypy translator test_list_setitem
9 Pypy translator test_list_slice
10 Pypy translator test_list_getitem
11 Pypy translator test_list_extend
12 Pybench ListSlicing
13 Pybench SmallLists
14 Tommti TommtiVector
15 Pybench SimpleListComprehensions
16 Pybench NestedListComprehensions
17 Pybench DictCreation
18 Pybench DictWithStringKeys
19 Pybench DictWithFloatKeys
20 Pybench DictWithIntegerKeys
21 Pybench SimpleDictManipulation
22 Tommti TommtiHashtest
23 Tommti TommtiHashes
24 Pypy translator test_dict_setitem1
25 Pypy translator test_dict_setitem2
26 Pypy translator test_dict_creation_mode1
27 Pypy translator test_dict_creation_mode2
28 Pypy translator test_dict_creation_mode3
29 Pypy translator test_dict_getitem
30 Pypy translator test_dict_raw_range
31 Pypy translator test_dict_class_dict_getmethod
32 Pypy translator test_dict_instance_getattr_instance_dict
33 Pypy translator test_dict_instance_setattr_instance_dict
34 Pypy translator test_dict_instance_setnewattr_instance_dict
35 Pypy translator test_dict_class_dict_getmethod_old_style
36 Pypy translator test_dict_instance_getattr_instance_dict_old_style
37 Pypy translator test_dict_instance_setattr_instance_dict_old_style
38 Pypy translator test_dict_instance_setnewattr_instance_dict_old_style
39 Pypy translator test_count_in_dict
Higher-order functions
1 Functional microbenchmarks LambdaCreation
2 Functional microbenchmarks HigherOrder
3 Functional microbenchmarks LambdaInvokation
4 Functional microbenchmarks FunctionCurrying
5 Functional microbenchmarks FunctionMap
6 Functional microbenchmarks FunctionFilter
7 Functional microbenchmarks FunctionReduce
I/O
Number Microbenchmark Test
1 JavaGrande IOLinkList
2 JavaGrande IOBinaryTree
3 JavaGrande IOVector
4 JavaGrande IOArray
5 Tommti TommtiIO
Threads and Processes
Number Microbenchmark Test
Threads
1 Custom ThreadCreation
2 Custom ThreadRunAndJoin
3 Custom ThreadRunAndJoinSynchronization
Process
1 Custom ProcessCall
2 Custom ProcessCheckCall
3 Custom ProcessCheckOutput
4 Custom ProcessPopen
Metaprogramming
Number Microbenchmark Test
Introspection
1 Metaprogramming Microbenchmarks - Introspection IntrospectionClassLookupBench
2 Metaprogramming Microbenchmarks - Introspection FunctionsInvoke
3 Metaprogramming Microbenchmarks - Introspection MethodsInvokeObject
4 Metaprogramming Microbenchmarks - Introspection MethodsInvokeClass
5 Metaprogramming Microbenchmarks - Introspection DynamicAttributeLookup
6 Metaprogramming Microbenchmarks - Introspection DynamicVariableLookup
7 Metaprogramming Microbenchmarks - Introspection DynamicAttributeWrite
8 Metaprogramming Microbenchmarks - Introspection DynamicVariableWrite
9 Pypy translator test_isinstance1
10 Pypy translator test_isinstance2
11 Pypy translator test_isinstance3
12 Pypy translator test_isinstance1_old_style
13 Pypy translator test_isinstance3_old_style
Structural Intercession
1 Metaprogramming Microbenchmarks - Intercession CreateClass
2 Metaprogramming Microbenchmarks - Intercession CreateClassWithInstance
3 Metaprogramming Microbenchmarks - Intercession CreateEmptyClass
4 Metaprogramming Microbenchmarks - Intercession CreateEmptyClassWithInstance
5 Metaprogramming Microbenchmarks - Intercession AttributeAddIntClass
6 Metaprogramming Microbenchmarks - Intercession AttributeAddObjectClass
7 Metaprogramming Microbenchmarks - Intercession AttributeAccessAddedClassInt
8 Metaprogramming Microbenchmarks - Intercession AttributeAccessAddedClassObject
9 Metaprogramming Microbenchmarks - Intercession AttributeAddIntObject
10 Metaprogramming Microbenchmarks - Intercession AttributeAddObjectObject
11 Metaprogramming Microbenchmarks - Intercession AttributeAccessAddedClassObject
12 Metaprogramming Microbenchmarks - Intercession AttributeAccessAddedObjectObject
13 Metaprogramming Microbenchmarks - Intercession AttributeDeleteIntClass
14 Metaprogramming Microbenchmarks - Intercession AttributeDeleteIntObject
15 Metaprogramming Microbenchmarks - Intercession AttributeDeleteObjectClass
16 Metaprogramming Microbenchmarks - Intercession AttributeDeleteObjectObject
17 Metaprogramming Microbenchmarks - Intercession MethodsAddClass
18 Metaprogramming Microbenchmarks - Intercession MethodsAddInvokeClass
19 Metaprogramming Microbenchmarks - Intercession MethodsAddObject
20 Metaprogramming Microbenchmarks - Intercession MethodsAddInvokeObject
21 Metaprogramming Microbenchmarks - Intercession MethodsDeleteAddedClass
22 Metaprogramming Microbenchmarks - Intercession MethodsDeleteAddedObject
23 Metaprogramming Microbenchmarks - Intercession TypeChange
24 Metaprogramming Microbenchmarks - Intercession InheritanceTreeChange
Behavioral Reflection
1 Metaprogramming Microbenchmarks - Computational Reflection GetAttrExistingBench
2 Metaprogramming Microbenchmarks - Computational Reflection GetAttrNonExistingBench
3 Metaprogramming Microbenchmarks - Computational Reflection SetAttrExistingBench
4 Metaprogramming Microbenchmarks - Computational Reflection SetAttrNonExistingBench
5 Metaprogramming Microbenchmarks - Computational Reflection HasAttrExistingBench
6 Metaprogramming Microbenchmarks - Computational Reflection HasAttrNonExistingBench
7 Metaprogramming Microbenchmarks - Computational Reflection CallBench
Characteristic: Dynamic code evaluation
1 Metaprogramming Microbenchmarks - Dynamic code execution EvalBench
2 Metaprogramming Microbenchmarks - Dynamic code execution ExecBench
3 Metaprogramming Microbenchmarks - Dynamic code execution ExecFileBench
4 Pypy translator test_dispatch_nop

Benchmarks

Common operations
Number Suite Benchmark
1 Unladen swallow CallMethod
2 Unladen swallow CallMethodSlots
3 Unladen swallow CallMethodUnknown
4 Unladen swallow CallSimple
5 Unladen swallow Float
6 Unladen swallow Logging
7 Unladen swallow LoggingSimpleOutput
8 Unladen swallow LoggingNoOutput
9 Unladen swallow NQueens
10 Unladen swallow Pidigits
11 Unladen swallow RegexCompile
12 Unladen swallow RegexEffbot
13 Unladen swallow RegexV8
14 Unladen swallow Richards
15 Unladen swallow ThreadCount
16 JavaGrande JGFCryptBench
17 JavaGrande JGFFFTBench
18 JavaGrande JGFHeapSortBench
19 JavaGrande JGFLUFactBench
20 JavaGrande JGFSeriesBench
21 JavaGrande JGFSparseMatmultBench
22 JavaGrande JGFMolDynBench
23 JavaGrande JGFRayTracerBench
24 JavaGrande JGFSearchBench
25 Crypto AESBench
26 Crypto MD5Bench
27 Jolden benchmarks TSPBench
28 Parrot benchmarks B1UnboundedRecursion
29 Parrot benchmarks B2GeneratePI
30 Parrot benchmarks B3Sorting
31 Parrot benchmarks B5StandardLibrary
32 Parrot benchmarks B6Iterators
33 Computer Shootout Language Benchmarks NBody
34 Computer Shootout Language Benchmarks BinaryTrees
35 Computer Shootout Language Benchmarks ChameneosRedux
36 Computer Shootout Language Benchmarks FannkuchRedux
37 Computer Shootout Language Benchmarks SpectralNorm
38 Computer Shootout Language Benchmarks Fasta
39 Computer Shootout Language Benchmarks MeteorContest
40 Pystone Benchmark Pystone
I/O
Number Suite Benchmark
1 Computer Shootout Language Benchmarks RegexDNA
2 Computer Shootout Language Benchmarks ReverseComplement
Multithreading
Number Suite Benchmark
1 Unladen swallow Multithreading
2 Unladen swallow UnpackSequence
3 Computer Shootout Language Benchmarks ThreadRing
Metaprogramming
Number Suite Benchmark
1 Unladen swallow Json
2 Unladen swallow Pickle
3 Unladen swallow PickleList
4 Unladen swallow PickleUnpickle
5 Unladen swallow PickleUnpickleList
6 Unladen swallow PickleDict
7 Unladen swallow JsonLoad
8 Parrot benchmarks B0Compiler
9 Parrot benchmarks B4Compiler
10 Intercessive Pybench instancesDynamic
11 Intercessive Pybench lookupsDynamic
12 Intercessive Pybench callsDynamic
13 Dynamic inheritance PybenchDynamicInheritance
14 Dynamic inheritance ShootoutDynamicInheritance
15 Dynamic inheritance WordcountDynamicInheritance
16 Dynamic inheritance ProductsManager

Programs

The following miscellaneous program suite comprises common, I/O and multithreading code.
Common operations
Number Program
1 AdatronSVM
2 MazeSolver
3 AntColonyOptimization
4 BarnesHut
5 BrainfuckInterpreter
6 ChaosgameFractals
7 Chess
8 RGBConverter
9 Dijkstra
10 DijkstraBidirectional
11 Genetic
12 Genetic2
13 KanoodlePuzzle
14 LinearAlgebra
15 LoopNodes
16 Mandelbrot2
17 AmbientOcclusionRenderer
18 Minilight
19 MaximumMatchingGraphs
20 ColorPatternsShells
21 OthelloGrame
22 PathTracer
23 SatSolver
24 RayTracer
25 Richards
26 RubikSolver
27 Score4Game
28 SHAImplementation
29 PrimesSieveOfAtkin
30 Sokoban
31 SudokuSolver1
32 SudokuSolver3
33 SudokuSolver4
34 Voronoi
35 Go
36 mandelbrotSimple
37 Mastermind2
38 NeuralNetwork
39 RubikSolver2
40 SolitaireEncryption
I/O
Number Program
1 ArithmeticCompressor_EncodeDecode
2 BlockCompression
3 Hq2xFilter
4 LempelZiv
5 NaturalLanguageParserPLCFRS
6 RSync
7 SudokuSolver2
8 JpegDecoder
Multithreading
Number Program
1 ConwayGameOfLife
2 SudokuSolver5
3 TicTacToe
4 Mastermind
Metaprogramming operations are represented by the BioPython program suite.
Metaprogramming
Number Program
1 test_Ace
2 test_AlignIO_FastaIO
3 test_AlignIO_convert
4 test_BioSQL
5 test_BioSQL_SeqIO
6 test_CAPS
7 test_Chi2
8 test_CodonTable
9 test_CodonUsage
10 test_Compass
11 test_Crystal
12 test_EmbossPrimer
13 test_Entrez
14 test_Enzyme
15 test_File
16 test_FSSP
17 test_GACrossover
18 test_GAOrganism
19 test_GAQueens
20 test_GARepair
21 test_GASelection
22 test_GenBank
23 test_HMMCasino
24 test_HMMGeneral
25 test_IsoelectricPoint
26 test_KEGG
27 test_KeyWList
28 test_Location
29 test_Medline
30 test_Motif
31 test_NCBIStandalone
32 test_NCBITextParser
33 test_NCBIXML
34 test_Nexus
35 test_NNExclusiveOr
36 test_NNGene
37 test_NNGeneral
38 test_PAML_baseml
39 test_PAML_codeml
40 test_PAML_yn00
41 test_ParserSupport
42 test_Pathway
43 test_Phd
44 test_Phylo
45 test_PhyloXML
46 test_PopGen_FDist_nodepend
47 test_PopGen_GenePop_nodepend
48 test_PopGen_SimCoal_nodepend
49 test_ProtParam
50 test_Restriction
51 test_SCOP_Astral
52 test_SCOP_Cla
53 test_SCOP_Des
54 test_SCOP_Dom
55 test_SCOP_Hie
56 test_SCOP_Raf
57 test_SCOP_Residues
58 test_SCOP_Scop
59 test_SeqIO
60 test_SeqIO_AbiIO
61 test_SeqIO_convert
62 test_SeqIO_FastaIO
63 test_SeqIO_features
64 test_SeqIO_index
65 test_SeqIO_QualityIO
66 test_SeqIO_SeqXML
67 test_SeqIO_write
68 test_SeqRecord
69 test_SeqUtils
70 test_Seq_objs
71 test_SffIO
72 test_SwissProt
73 test_translate
74 test_Tutorial
75 test_Uniprot
76 test_align
77 test_bgzf
78 test_geo
79 test_pairwise2
80 test_prodoc
81 test_prosite1
82 test_prosite2
83 test_seq
84 test_PopGen_SimCoal_nodepend
85 test_UniGene

Large Scale Applications

Clicking on each application shows the detailed profiler output of each application, remarking the major code category used in each call.
Common Operations
Number Application
1 Bazaar
2 Cog
3 Html5Lib
4 Mako
5 Spitfire
I/O
Number Application
1 2to3
2 BlindElephant
3 Pyramid Web Framework
4 Volatility Framework
Multithreading
Number Application
1 SpamBayes
2 SQLMap
3 Web2Py Web framework
Metaprogramming
Number Application
1 Django
2 Rietveld