FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Plugin models...

 
Post new topic   Reply to topic    PyLucid - CMS - Forum Forum Index -> system
View previous topic :: View next topic  
Author Message
jens
Administrator


Joined: 12 Oct 2005
Posts: 972
Location: duisburg, germany

PostPosted: Fri 30 May, 2008 05:52    Post subject: Plugin models... Reply with quote

Some plugins needs his own models!

Started with the new dynamic sub menu i needed a way to store some stuff for the sorting. I used the plugin preferences for this and put a dictionary as a preference item. For this small data, it's ok...

With the new flv player i have the same problem. I need a way to store meta data for _every_ existing flv file. This can be mutch more data, than a small dict.

If someone would like to create a Blog, Forum or Image Gallery Plugin, he will have also the need to store data. These plugins needs his own models.

The only existing way is, to add a Plugin as a django app in settings.INSTALLED_APPS. I made this in a PyRM proof of concept and it works. After a syncdb, the model tables created and the plugin can use this own database tables.

The bad thing with settings.INSTALLED_APPS: It hasn't been related to the PyLucid plugin administator. If the user "uninstall" a plugin, the plugin models would not removes from the database.
The good thing is: It's existing. Nothing to implement Wink

Other Ideas:

1. Could we easy implement a "register" mechanism to the existing model stuff? And would these models works equal to normal django models?
Explain: In the plugin config class we add something like e.g. models.register("MyModels")...

2. Can the existing PyLucid models package import all plugin models?

3. Can we use MonkeyPatching here? (But i thing this is not the best way)

Any comments?

EDIT: For geman guys, i created a cross post here: http://www.python-forum.de/topic-14848.html (de)

EDIT2: Hm. Did i think too complicated? Could we create a models class somewhere and use it, if the related tables exists?
_________________

http://www.jensdiemer.de | http://www.htfx.de | http://www.python-forum.de
Back to top
View user's profile Send private message
rantaaho



Joined: 09 Jan 2008
Posts: 47
Location: Kuopio, Finland

PostPosted: Fri 30 May, 2008 07:55    Post subject: Reply with quote

Currently, auth "plugin" has it's own model for passwords. So, I have been thinking these same things when I was planning how to make authentication more "plugin-like".

There seems to be two contradictory requirements, need for new django apps with their own models and on the other side one should be able to install and use PyLucid without shell access.

Modifying settings.py with PyLucid plugin administator is IMHO a bad idea from security point of view.

Could one solution be: Split PyLucid to two django apps 1) PyLucid 2) Plugins. The first one would contain only basic and all Plugins would go to the second "app". Then each plugin (Plugins/Foo/, Plugins/Bar/ etc.) would provide it's own models and views. And in Plugins/models we would do some magic to import them all? This is close to your idea #2, but IMHO little bit cleaner since PyLucid/models would not be affected.

The good thing with this would be that all plugins would also be close to normal django apps, they would just in addition to normal stuff have the PyLucid Plugin API to play with.

In my opinion #1 is too complicated, #3 a bad idea and EDIT2 maybe simpler to implement but conceptually more complicate as it is braking typical djando behaviour.
Back to top
View user's profile Send private message
jens
Administrator


Joined: 12 Oct 2005
Posts: 972
Location: duisburg, germany

PostPosted: Fri 30 May, 2008 15:05    Post subject: Reply with quote

rantaaho wrote:
There seems to be two contradictory requirements, need for new django apps with their own models and on the other side one should be able to install and use PyLucid without shell access.

Thats not the problem. If you add a other django app with own models in your settings.py, you can use the PyLucid _install section to create the tables with syncdb. It will create all tables for all apps and not only for the PyLucid models.

rantaaho wrote:
Modifying settings.py with PyLucid plugin administator is IMHO a bad idea from security point of view.

Yes, this was no solution for me.

rantaaho wrote:
Could one solution be: Split PyLucid to two django apps 1) PyLucid 2) Plugins. The first one would contain only basic and all Plugins would go to the second "app". Then each plugin (Plugins/Foo/, Plugins/Bar/ etc.) would provide it's own models and views. And in Plugins/models we would do some magic to import them all? This is close to your idea #2, but IMHO little bit cleaner since PyLucid/models would not be affected.

The good thing with this would be that all plugins would also be close to normal django apps, they would just in addition to normal stuff have the PyLucid Plugin API to play with.

Hm. Change PyLucid Plugins more to normal django apps can be a good idea. But this is more a big thing for the feature. We must refactor every plugin. I think this is to mutch work, now.

rantaaho wrote:
... EDIT2 maybe simpler to implement but conceptually more complicate as it is braking typical djando behaviour.

On the other Side, PyLucid breaks many typical django behaviour at the moment Wink

It could look like this:
* In the plugin install routine we implement this: Get the CREATE TABLE statement for all plugin models and execute it.
* In the deinstall routine we add a function to DROP all plugin model tables

We can enhanced the deinstall routine and can ask the admin, if the created tables should be droped or not. And we can insert a backup funtion for the plugin preferences. On the other side, if a admin will deinstall a plugin, he normaly would like to have a clean database, isn't it?

The question is, worked "seperate" models like the normal one? What is needed to init these models? How mutch work is it?
_________________

http://www.jensdiemer.de | http://www.htfx.de | http://www.python-forum.de
Back to top
View user's profile Send private message
jens
Administrator


Joined: 12 Oct 2005
Posts: 972
Location: duisburg, germany

PostPosted: Fri 30 May, 2008 16:54    Post subject: Reply with quote

Great!

I made a test and it seems to work:
1. I insert a test model into a existing model file (e.g. ./models/Style.py).
2. run syncdb in the install section created the table
3. move the model class from Style.py into a plugin
4. try to play with the model

EDIT: The question now is, can i get the CREATE TABLE statement from a seperated model class? I look at this now...

EDIT2: Greater! It's all very easy! We must only import the plugin models and after this it's "appends" to the PyLucid app. So syncdb would create the tables for us.
Installing a plugin can work like this: After the current routine installed the plugin into the database, we must import the plugin or the model file and run syncdb. Thats all...

OK. The question is no more, if it will work Smile

EDIT3: Here you can see, syncdb will recognizes the models:
http://trac.pylucid.net/browser/trunk/dev_scripts/local_tests/models_test.py?rev=1606
_________________

http://www.jensdiemer.de | http://www.htfx.de | http://www.python-forum.de
Back to top
View user's profile Send private message
jens
Administrator


Joined: 12 Oct 2005
Posts: 972
Location: duisburg, germany

PostPosted: Fri 30 May, 2008 20:44    Post subject: Reply with quote

It's a little bit tricky Sad

There is no problem to get the CREATE TABLE statements:
Code:
from django.core.management.color import no_style
from django.core.management import sql
style = no_style()

def get_create_table(models):
    statements = []
    for model in models:
        statements += sql.sql_model_create(model, style)[0]
        statements += sql.sql_indexes_for_model(model, style)
        statements += sql.custom_sql_for_model(model)
    return statements

models is a list of all model classes. This is no problem, if the plugin defined a list of all classes.


But the problem is to get the DROP TABLE statements. The only thing i found is this:

Code:
from django.core.management import sql
from django.db.models import get_app
from django.core.management.color import no_style

style = no_style()
app = get_app("PyLucid")
statements = sql.sql_delete(app, style)

But here we get all DROP statements for all models from the app.

Everything around creating and drop tables where easy, if every plugin is a own INSTALLED_APPS entry Sad

And now?

Can we expand INSTALLED_APPS at runtime? (Not saving it into settings.py file) We need this IMHO only, to get the sql statements and not if the plugin method called from the plugin manager...
_________________

http://www.jensdiemer.de | http://www.htfx.de | http://www.python-forum.de
Back to top
View user's profile Send private message
jens
Administrator


Joined: 12 Oct 2005
Posts: 972
Location: duisburg, germany

PostPosted: Fri 30 May, 2008 22:06    Post subject: Reply with quote

Manipulating INSTALLED_APPS at runtime works:

http://trac.pylucid.net/changeset/1607

I add a fake django app "PyLucid.system.PyLucidPlugins" and add register models at runtime to this app. After getting DROP TABLES statements, i delete the added models from the app cache...

Now i have a second idea. You wrote about adding a second plugin... We can add the fake django app "PyLucid.system.PyLucidPlugins" permanently?
We only use django.db.models.loading.register_models to add the plugin models and del(django.db.models.loading.cache.app_models["PyLucidPlugins"]) to remove the models...
We should test if we can do that in a threaded environment? On the other side, we only need this hack for getting the delete table statements...
_________________

http://www.jensdiemer.de | http://www.htfx.de | http://www.python-forum.de
Back to top
View user's profile Send private message
jens
Administrator


Joined: 12 Oct 2005
Posts: 972
Location: duisburg, germany

PostPosted: Sat 31 May, 2008 14:18    Post subject: Reply with quote

OK. It works. I added a faked plugin permanetly.

But there is a problem with unittest:

I tried to init the plugin and the models in setUP and remove them in tearDown. Than i get something like this:
Code:
[u'PyLucid_js_logindata', u'PyLucid_page', u'PyLucid_pagearchiv', u'PyLucid_plugin', u'PyLucid_style', u'PyLucid_template', u'PyLucid_testalbum', u'PyLucid_testartist', u'auth_group', u'auth_group_permissions', u'auth_message', u'auth_permission', u'auth_user', u'auth_user_groups', u'auth_user_user_permissions', u'django_admin_log', u'django_content_type', u'django_session', u'django_site']
'DROP TABLE "PyLucid_testartist";'
'DROP TABLE "PyLucid_testalbum";'
.Error: Database :memory: couldn't be flushed. Possible reasons:
      * The database isn't running or isn't configured correctly.
      * At least one of the expected database tables doesn't exist.
      * The SQL was invalid.
    Hint: Look at the output of 'django-admin.py sqlflush'. That's the SQL this command wasn't able to run.
    The full error: no such table: PyLucid_testalbum

Only a limit from SQLite :memory: ?

Then i tried to install the unittest plugin in the unittest init process, before we make the fixtures. Than i get this:
Code:
Loading PyLucid database dump 
Installing PyLucid internal plugins 
initialize the unittest plugin
copy tests/unittest_plugin/ to PyLucid/plugins_external/unittest_plugin
Install PyLucid.plugins_external.unittest_plugin... OK, ID: 24
'CREATE TABLE "PyLucid_testartist" (\n    "id" integer NOT NULL PRIMARY KEY,\n    "name" varchar(100) NOT NULL\n)\n;'
'CREATE TABLE "PyLucid_testalbum" (\n    "id" integer NOT NULL PRIMARY KEY,\n    "artist_id" integer NOT NULL,\n    "name" varchar(100) NOT NULL,\n    "num_stars" integer NOT NULL,\n    "createtime" datetime NOT NULL,\n    "lastupdatetime" datetime NOT NULL,\n    "createby_id" integer NULL,\n    "lastupdateby_id" integer NULL\n)\n;'
'CREATE INDEX "PyLucid_testalbum_artist_id" ON "PyLucid_testalbum" ("artist_id");'
'CREATE INDEX "PyLucid_testalbum_createby_id" ON "PyLucid_testalbum" ("createby_id");'
'CREATE INDEX "PyLucid_testalbum_lastupdateby_id" ON "PyLucid_testalbum" ("lastupdateby_id");'
Building test suite.
Running tests
Problem installing fixture '/tmp/PyLucid_tFnguv.json': columns app_label, model are not unique
EProblem installing fixture '/tmp/PyLucid_tFnguv.json': columns app_label, model are not unique
EProblem installing fixture '/tmp/PyLucid_tFnguv.json': columns app_label, model are not unique
EProblem installing fixture '/tmp/PyLucid_tFnguv.json': columns app_label, model are not unique
EProblem installing fixture '/tmp/PyLucid_tFnguv.json': columns app_label, model are not unique
EProblem installing fixture '/tmp/PyLucid_tFnguv.json': columns app_label, model are not unique
E
======================================================================
ERROR: test_first_check (tests.test_plugin_api.PluginAPI_TestCase)



I commit the current state witch makes the trunk a little brocken Wink
http://trac.pylucid.net/changeset/1609
_________________

http://www.jensdiemer.de | http://www.htfx.de | http://www.python-forum.de
Back to top
View user's profile Send private message
jens
Administrator


Joined: 12 Oct 2005
Posts: 972
Location: duisburg, germany

PostPosted: Sat 31 May, 2008 23:09    Post subject: Reply with quote

The two errors was a concept failure by me. I made two errors Wink
1. The app_label of a plugin model must be 'PyLucidPlugins' and not "PyLucid"!
2. The test plugin would be copied to plugins_internal instead to plugins_external so this plugin would be auto installed with all other plugins and goes into the fixtures.

With http://trac.pylucid.net/changeset/1612 everything around the new "plugin models" should be run fine! I have updated the unittest for this and test many cases.

With these changes from the last days, we get a very cool features in the PyLucid plugin systems.

Everything looks great, isn't it?

So now we must create docu about all this stuff Wink

EDIT: Add a smal Docu about this: http://www.pylucid.org/_goto/163/plugin-models/

EDIT2: A other simple example for plugin models: The new page counter: http://trac.pylucid.net/changeset/1613 Smile
_________________

http://www.jensdiemer.de | http://www.htfx.de | http://www.python-forum.de
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    PyLucid - CMS - Forum Forum Index -> system All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

<< back to PyLucid CMS Homepage



Powered by phpBB © 2001, 2005 phpBB Group