La prezentarea trecuta RONUA Andrei Rinea a vorbit despre GEO IP lookup pentru aflarea locatiei vizitatorilor.
Motivat de faptul ca Andrei Rinea este un perfectionist si a facut doua cautari, am incercat sa fac si eu o (asa zisa) imbunatatire.
Mai intii am vrut sa citesc mai usor din csv-urile respective.Sa zicem ca vreau sa citesc asta:
Copyright (c) 2011 MaxMind Inc. All Rights Reserved.
startIpNum,endIpNum,locId
“16777216”,”16777471″,”17″
“16777472”,”16778239″,”49″
……
Am downloadat FileHelpers si am scris codul:
[DelimitedRecord(",")]
public class IPRangeAndrei
{
[FieldQuoted()] public long Start;
[FieldQuoted()] public long End;
[FieldQuoted()] public int LocID;
}
public class FileHelperImporter
{
public IEnumerable<GeoLocation> ImportLocations(string csvFilePath, int HeaderLines=2)
{
DelimitedFileEngine engine = new DelimitedFileEngine(typeof (GeoLocation));
engine.Options.IgnoreFirstLines = HeaderLines;
return engine.ReadFile(csvFilePath) as GeoLocation[];
}
public IEnumerable<IPRangeAndrei> ImportIP(string csvFilePath, int HeaderLines=2)
{
DelimitedFileEngine engine = new DelimitedFileEngine(typeof(IPRangeAndrei));
engine.Options.IgnoreFirstLines = HeaderLines;
return engine.ReadFile(csvFilePath) as IPRangeAndrei[];
}
}
Pare un pic mai clar . Din pacate, e si mai incet – 2 secunde fata de 1 secunda la un fisier de 117 MB .
Apoi am vrut sa incar la compile time csv-urile, nu la runtime.
Prima oara am incarcat cele de GeoLiteCity-Location.csv – puteti sa le luati de aici : http://www.maxmind.com/app/geolitecity
Am scris urmatorul fisier .tt
<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".cs" #>
<#@ assembly name="C:\Users\andrei\Desktop\g\Module\Module\bin\Release\Module.dll" #>
<#
Module.CsvImporter importer= new Module.CsvImporter ();
var locations = importer.ImportLocations(@"C:\Users\andrei\Desktop\GeoLiteCity_20111004\GeoLiteCity-Location.csv");
#>
using System.Collections.Generic;
using Module;
namespace AndreiTT
{
public class Location:Dictionary<string, GeoLocation>
{
public string GetFileKey(long Key)
{
return "K" + Key;
}
public Location():base()
{
<#
foreach(var geo in locations)
{
#>
this.Add(GetFileKey(<#=geo.Id#>),new GeoLocation(<#=geo.Id#>, "<#=geo.CountryCode#>", "<#=geo.RegionCode#>", "<#=geo.City#>", "<#=geo.PostalCode#>", <#=geo.Latitude#>f, <#=geo.Longitude#>f));
<#
}
#>
}
}
}
si a iesit fisierul asta ( de vreo 30 MB)
using System.Collections.Generic;
using Module;
namespace AndreiTT
{
public class Location:Dictionary<string, GeoLocation>
{
public string GetFileKey(long Key)
{
return "K" + Key;
}
public Location():base(320000)
{
this.Add(GetFileKey(1),new GeoLocation(1, "O1", "", "", "", 0f, 0f));
this.Add(GetFileKey(2),new GeoLocation(2, "AP", "", "", "", 35f, 105f));
this.Add(GetFileKey(3),new GeoLocation(3, "EU", "", "", "", 47f, 8f));
this.Add(GetFileKey(4),new GeoLocation(4, "AD", "", "", "", 42.5f, 1.5f));
this.Add(GetFileKey(5),new GeoLocation(5, "AE", "", "", "", 24f, 54f));
Inspirat de aceasta reusita, am facut acelasi lucru si pentru GeoLiteCity-Blocks.
Singura problema a fost ca a dat un fisier de 194 MB – care , la compilare cu Visual studio a dat:
Source file ‘C:\Users\andrei\Desktop\g\App\AndreiTT\Blocks.cs’ could not be opened (‘Unspecified error ‘)
OK, atunci l-am compilat cu msbuild – a dat:
MSBUild: CSC : error CS0010: Unexpected fatal error – ‘a’
M-am uitat la
http://msdn.microsoft.com/en-us/library/bb546037%28v=vs.90%29.aspx
Si zice:
Unexpected fatal error — ‘error’.
This error is generated when something completely unexpected occurs to stop compilation.
Asa ca l-am spart pe blocks.tt in 5 bucati, fiecare de 1.000.000 de bucati si a generat fisiere de 30- 55 MB . A generat partial class, asta ca sa le puna pe toate la un loc:
Asa ca msbuild a dat o eroare omeneasca
(CoreCompile target) ->
CSC : error CS0010: Unexpected fatal error — ‘Not enough storage is available to process this command. ‘ [C:\Users\andrei\Desktop\g\App\AndreiTT\AndreiTT.c
sproj]
0 Warning(s)
1 Error(s)
Time Elapsed 00:03:34.15
Ok, daca nu poate sa asambleze mai multe .tt din fisiere diferite, atunci sa le pun in dll-uri diferite . Am facut si asta.
A generat 5 dll-uri, fiecare de cam cit 20 MB.
Le-am referentiat , am scris codul sa caute in fiecare dll si … la rulare OutOfMemoryException.
Andrei Rinea ma prevenise – asa ca i-am dat telefon sa ii spun ca e asa cum zice el. M-a intrebat daca am x64. Asa ca … l-am compilat si pe x64. La rulare mi-a inghitit toti cei 4GB de RAM si a blocat PC-ul.
Asa incit fiti prudenti – compile time nu e intotdeauna mai bun decat runtime !
Daca vreti sa experimentati, gasiti aici un proiect cu care sa va jucati :
http://msprogrammer.serviciipeweb.ro/wp-content/uploads/runcomp.7z
