Python Saving Interest Rate Curve Objects to File

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Python Saving Interest Rate Curve Objects to File

TSchulz85
Hi,

I'm very new to QuantLib and Python. Currently I'm bootstrapping e.g. EONIA curves for different dates in the past. These curves I'm saving in Dict. Now I'm trying to save this Dict into a file, so I can access the historical curves without doing the full bootstrapping process (and manually adjusting some of the input datapoints).
I've tried to pickle it and also tried different other methods (HDF5).
e.g.:

import pickle

Eoniafile = open('eoniaDB.obj', 'w')
pickle.dump(EoniaDB, Eoniafile)

This returns the following error:
TypeError: can't pickle SwigPyObject objects

Is there any way to save QuantLib curve objects to a drive or database?

Thanks,
Tobias
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

alihassani
There is a member of the swig curves called .nodes ()

You can try saving those and recreating the curve with those nodes.

On September 26, 2016 6:48:36 AM EDT, TSchulz85 <[hidden email]> wrote:
Hi, 

I'm very new to QuantLib and Python. Currently I'm bootstrapping e.g. EONIA
curves for different dates in the past. These curves I'm saving in Dict. Now
I'm trying to save this Dict into a file, so I can access the historical
curves without doing the full bootstrapping process (and manually adjusting
some of the input datapoints).
I've tried to pickle it and also tried different other methods (HDF5).
e.g.:

import pickle

Eoniafile = open('eoniaDB.obj', 'w')
pickle.dump(EoniaDB, Eoniafile)

This returns the following error:
TypeError: can't pickle SwigPyObject objects

Is there any way to save QuantLib curve objects to a drive or database?

Thanks,
Tobias



--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751.html
Sent from the quantlib-users mailing list archive at Nabble.com.





QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
------------------------------------------------------------------------------

_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Hi Ali,
Thanks a lot for your answer. Ideally I would like to just save the actual curves, as this would avoid bootstrapping all the historical curves for all the different currencies every day (or week) for the past x years.
In case this would be just not possible, your solution might be a good workaround. I guess you would just save those values in a dict, list or similar and then bootstrap with those values? Is there an easy way to do the bootstrapping just based on the nodes?
But ideally I would be able to just save the actual curves.
I know it is a little more of a Python question than a QuantLib question. But as I assume in most Python forums, people can not really replicate the problem and maybe someone here already encountered a similar problem. But I guess it would be also relevant for other people who would like to use QuantLib for backtesting strategies, charting etc.
Thanks,
Tobias
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

alihassani
I don't think its possible to serialize/save swig objects.

In my humble opinion, I don't think recreating the curve from the nodes is as expensive an operation as bootstrapping the curve from scratch. But I guess you can test it.

On September 26, 2016 7:27:46 AM EDT, TSchulz85 <[hidden email]> wrote:
Hi Ali, 
Thanks a lot for your answer. Ideally I would like to just save the actual
curves, as this would avoid bootstrapping all the historical curves for all
the different currencies every day (or week) for the past x years.
In case this would be just not possible, your solution might be a good
workaround. I guess you would just save those values in a dict, list or
similar and then bootstrap with those values? Is there an easy way to do the
bootstrapping just based on the nodes?
But ideally I would be able to just save the actual curves.
I know it is a little more of a Python question than a QuantLib question.
But as I assume in most Python forums, people can not really replicate the
problem and maybe someone here already encountered a similar problem. But I
guess it would be also relevant for other people who would like to use
QuantLib for backtesting strategies, charting etc.
Thank s,
Tobias



--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751p17753.html
Sent from the quantlib-users mailing list archive at Nabble.com.





QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
------------------------------------------------------------------------------

_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Thanks a lot Ali!
Is there a good way to recreate (lets say the eonia and euribor) curve with the nodes or would you just bootstrap it again with those rates. Which RateHelpers would you use in that case?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

Luigi Ballabio

As an alternative, you can convert the dates to a format you can serialize; for instance, you can convert them to Python datetime objects or extract their serial numbers. When you read them back, you'll convert them back to QuantLib dates and rebuild the curve.

Luigi


On Mon, Sep 26, 2016, 20:08 TSchulz85 <[hidden email]> wrote:
Thanks a lot Ali!
Is there a good way to recreate (lets say the eonia and euribor) curve with
the nodes or would you just bootstrap it again with those rates. Which
RateHelpers would you use in that case?



--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751p17755.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

------------------------------------------------------------------------------

_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Hi,

I managed to save the nodes into a file with the method you suggested (converting to datetime).
What would be now the best method to recreate the curves with the nodes? Is there a ready-to-go method or should I for example convert the discount factors into an equivalent of e.g. depo rates and then bootstrap the curve again?

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

Luigi Ballabio
You can instantiate an interpolated curve. The actual type depends on which kind of curve you bootstrapped to begin with. If you used a PiecewiseFlatForward and you stored the (date, forward) nodes, you can use them to create a ForwardCurve instance. If you bootstrapped a curve that interpolates discounts or zero yields, you might have to add the corresponding class to the SWIG interfaces and export it.  In general: first, look at piecewiseyieldcurve.i and find out what kind of curve you used; for instance, the lines 

    export_piecewise_curve(PiecewiseFlatForward,ForwardRate,BackwardFlat);
    export_piecewise_curve(PiecewiseLogCubicDiscount,Discount,MonotonicLogCubic);

tell you that PiecewiseFlatForward uses BackwardFlat interpolation on (instantaneous) forward rates, and PiecewiseLogCubicDiscount uses MonotonicLogCubic interpolation on discounts.  Then look into forwardcurve.i if you're interpolating forwards, discountcurve.i for discounts and zerocurve.i for zero rates. If you find a line in there corresponding to the interpolation you used for the bootstrap, you're all set; for instance, if you used PiecewiseFlatForward, you'll see 

    export_forward_curve(ForwardCurve,BackwardFlat);

that tells you that you can use ForwardCurve, which has the same interpolation.  If there's no such line, you'll have to add it; for instance, if you used PiecewiseLogCubicDiscount, you'll have to add a line like

    export_discount_curve(MyDiscountCurve,MonotonicLogCubic);

to discount.i and rebuild the interfaces (MonotonicLogCubic is the interpolation; MyDiscountCurve is the name that you want to use for the curve and you can change it to whatever you want).

Hope this helps,
    Luigi


On Mon, Sep 26, 2016 at 8:51 PM TSchulz85 <[hidden email]> wrote:
Hi,

I managed to save the nodes into a file with the method you suggested
(converting to datetime).
What would be now the best method to recreate the curves with the nodes? Is
there a ready-to-go method or should I for example convert the discount
factors into an equivalent of e.g. depo rates and then bootstrap the curve
again?





--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751p17757.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users


------------------------------------------------------------------------------

_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Both the eonia and euribor curve are build as PiecewiseLogCubicDiscount, it is almost exactely following your example from the Python QuantLib Cookbook.
I manages to 'rebuild' the curve as flat forward with
DiscountCurve(QDate, nodes, Actual365Fixed(), TARGET())

However, I would like to rebuild it again as PiecewiseLogCubicDiscount. How would i best do that in Python?

When you're refering to the ".i" and the "export_piecewise_curve", "export_forward_curve" and "export_discount_curve", are those C++ functions? I can't seem to find those in the Python QuantLib.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

Luigi Ballabio
You should add to the Python module a LogCubicDiscountCurve and use that one.  The .i files I was referring to are in the QuantLib-SWIG/SWIG folder, and the "export_..." are SWIG macros defined in those files.  You'll have to add 

export_discount_curve(LogCubicDiscountCurve,MonotonicLogCubic);

to discountcurve.i and recompile the module.

Hope this helps,
    Luigi


On Wed, Sep 28, 2016 at 2:28 PM TSchulz85 <[hidden email]> wrote:
Both the eonia and euribor curve are build as PiecewiseLogCubicDiscount, it
is almost exactely following your example from the Python QuantLib Cookbook.
I manages to 'rebuild' the curve as flat forward with
DiscountCurve(QDate, nodes, Actual365Fixed(), TARGET())

However, I would like to rebuild it again as PiecewiseLogCubicDiscount. How
would i best do that in Python?

When you're refering to the ".i" and the "export_piecewise_curve",
"export_forward_curve" and "export_discount_curve", are those C++ functions?
I can't seem to find those in the Python QuantLib.



--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751p17769.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

------------------------------------------------------------------------------

_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Thanks a lot, that almost worked perfectly. Perfomance is extremly fast and and (usually) matches the original curve perfectly.
However, i've recognised it doesn't work for the jumps. Is there any good way to include them as well?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

Luigi Ballabio
You're correct.  You'll have to add to the SWIG interfaces an overloaded constructor that takes the jumps (which is already available in C++) and recompile yet again.

Luigi




On Tue, Oct 4, 2016 at 2:28 PM TSchulz85 <[hidden email]> wrote:
Thanks a lot, that almost worked perfectly. Perfomance is extremly fast and
and (usually) matches the original curve perfectly.
However, i've recognised it doesn't work for the jumps. Is there any good
way to include them as well?



--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751p17784.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Thanks!
How do I add an overloaded constructor? Unfortunately my programming knowledge is very limited.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

Luigi Ballabio
In SWIG/discountcurve.i, try adding

        Name##Ptr(const std::vector<Date>& dates,
                  const std::vector<DiscountFactor>& discounts,
                  const DayCounter& dayCounter,
                  const Calendar& calendar,
                  const std::vector<Handle<Quote> >& jumps,
                  const std::vector<Date>& jumpDates,
                  const Interpolator& i = Interpolator()) {
            return new Name##Ptr(
                new InterpolatedDiscountCurve<Interpolator>(dates,discounts,
                                                            dayCounter,calendar,
                                                            jumps, jumpDates, i));
        }

immediately after the other similar declaration. Disclaimer: I haven't tested it.

Luigi


On Tue, Oct 4, 2016 at 3:57 PM TSchulz85 <[hidden email]> wrote:
Thanks!
How do I add an overloaded constructor? Unfortunately my programming
knowledge is very limited.



--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751p17786.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Just tried and it works perfectly, thanks!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
This post was updated on .
I'm actually experiencing a weird problem now. So in the meantime I've build a database with the knots for historical dates (no jump/jump dates) which I save into a dict and then into a pickle file (just as discussed above). Rebuilding the curves (eonia and euribor) works perfectly fine and all knots are identical.
I've used "export_discount_curve(LogCubicDiscountCurve,MonotonicLogCubic);" in the .i file

The unusal thing occures when pricing a forward swap. The original curves priced it fine, but for short swap starts i'm getting errors, e.g. (curve date as of 5-Oct-2010)

for a 1y1y fwd swap:

  File "C:\Users\tschulz_ldg\AppData\Local\Continuum\Anaconda2\lib\site-packages\QuantLib\QuantLib.py", line 14675, in fairRate
    return _QuantLib.VanillaSwap_fairRate(self)

RuntimeError: result not available

for a 6y1y fwd swap:
return _QuantLib.VanillaSwap_fairRate(self)

RuntimeError: 2nd leg: Missing Euribor6M Actual/360 fixing for October 5th, 2016

and a 7y1y fwd swap works fine.

however if i bootstrap a totally random new curve (PiecewiseLogCubicDiscount(...)) under another name, I can suddenly price the 6y1y, but not the 1y1y (pls not with the curves unaltered).
When I bootstrap the curve with the origianl values, but save under a new name, then suddenly the other curves (unaltered) even price the 1y1y.

Before posting all kind of code snippets I thought to first get some feedback what the issue could be.

In case the description above was too confusion, let me sumarize the behaviour:
1. load and rebuild "eonia_curve" and "euribor_curve" from pickle
2. fwd swap pricing for 1y1y, 7y1y (and many others doesnt work)
3. bootstrap a random curve names "curve1", fwd pricing of the 6y1y suddenly works
4. bootstrap the curves from 1. under new named "eonia_cuvre2" and "euribor_curve2" but as originally bootstrapped and not by rebuilding of the knots, suddenly even the 1y1y prices

Please let me know any ideas, happy to post more background, but not quite sure where to start.






Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
Okay, i think I found out my mistake, forgot to set

Settings.instance().evaluationDate = Date

I guess that just tells QL what day 'today' is?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

Luigi Ballabio
Yes, it does.

On Tue, Oct 11, 2016 at 5:41 PM TSchulz85 <[hidden email]> wrote:
Okay, i think I found out my mistake, forgot to set

Settings.instance().evaluationDate = Date

I guess that just tells QL what day 'today' is?



--
View this message in context: http://quantlib.10058.n7.nabble.com/Python-Saving-Interest-Rate-Curve-Objects-to-File-tp17751p17794.html
Sent from the quantlib-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
QuantLib-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/quantlib-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Python Saving Interest Rate Curve Objects to File

TSchulz85
This post was updated on .
Thanks!

Just for future reference if people are interested, here the main part of the discussed code:


#%% Creating the Database/bootstrapping and converting into pickle friendly Dict
EoniaDB = {}
EuriborDB = {}

for date in dates:
    [create helpers etc for date]
    eonia_curve = PiecewiseLogCubicDiscount(0, TARGET(), helpers, Actual365Fixed(), [] ,[] , 1e-8)
    euribor_curve = PiecewiseLogCubicDiscount(2, TARGET(), helpers+synth_helpers, Actual365Fixed())
   
    nodes = eonia_curve.nodes()
    dates = [dt.datetime(nodes[i][0].year(), nodes[i][0].month(), nodes[i][0].dayOfMonth()) for i in range(len(nodes))]
    rates = [nodes[i][1] for i in range(len(nodes))]
    EoniaDB[date] = zip(dates, rates)
   
    nodes = euribor_curve.nodes()
    dates = [dt.datetime(nodes[i][0].year(), nodes[i][0].month(), nodes[i][0].dayOfMonth()) for i in range(len(nodes))]
    rates = [nodes[i][1] for i in range(len(nodes))]    
    EuriborDB[date] = zip(dates, rates)

   
#%% Saving Database into pickle file

def save_obj(obj, name):
    with open('obj/'+ name + '.pkl', 'wb') as f:
        pickle.dump(obj, f, pickle.HIGHEST_PROTOCOL)

obj = EuriborDB
name = 'EuriborDB'
save_obj(obj, name)

obj = EoniaDB
name = 'EoniaDB'
save_obj(obj, name)


#%% Loading Database from pickle file

def load_obj(name):
    with open('obj/' + name + '.pkl', 'rb') as f:
        return pickle.load(f)

name = 'EuriborDB2'
LoadedEuriborDB = load_obj(name)

name = 'EoniaDB2'
LoadedEoniaDB = load_obj(name)


#%% Rebuilding Curve for a specific date
 LoadedPyNodes = LoadedEuriborDB[date]
 LoadedQDate = [Date(LoadedPyNodes[i][0].day, LoadedPyNodes[i][0].month, LoadedPyNodes[i][0].year) for i in range(len(LoadedPyNodes)) ]
 LoadedRates = [LoadedPyNodes[i][1] for i in range(len(LoadedPyNodes))]
 euribor_curve2 = LogCubicDiscountCurve(LoadedQDate, LoadedRates, Actual365Fixed(), TARGET())    
       
 LoadedPyNodes = LoadedEoniaDB[date]
 LoadedQDate = [Date(LoadedPyNodes[i][0].day, LoadedPyNodes[i][0].month, LoadedPyNodes[i][0].year) for i in range(len(LoadedPyNodes)) ]
 LoadedRates = [LoadedPyNodes[i][1] for i in range(len(LoadedPyNodes))]
 eonia_curve2 = LogCubicDiscountCurve(LoadedQDate, LoadedRates, Actual365Fixed(), TARGET())
Loading...