Welcome, Guest. Please login or register.
Did you miss your activation email?
April 25, 2014, 03:50:46 AM

Login with username, password and session length
Search:     Advanced search
Interested in joining the WebsiteBaker team?
For more Information read here or on our new website.
177514 Posts in 24328 Topics by 6879 Members
Latest Member: dorstcom
* Home Help Search Login Register
Pages: [1] 2   Go Down
Print
Author Topic: New Module: Curl Module (now with multi language support)  (Read 17181 times)
moulin

Offline Offline

Posts: 32


« on: April 21, 2006, 12:46:21 PM »

Curl Module
I made a small curl module for websitebaker. It is an initial release and some more options will be added.

What is Curl:
This Curl module almost works like a wrapper page. Instead of loading a full external page you can only show a selection with Curl. For example if you want to copy (with permission) the event agenda of your sports club which is on the homepage you can show the agenda and not all the text around it. Just specify the page where you want your get your information from. Look in the html file where the part starts that you need. Fill this in for the start and end. Voila (Frech for wow that is easy), the snippit will appear on your WebsiteBaker page.

How to use:
  • Install module  rolleyes
  • Make a Curl page
example:
  • Add the desired website. eg.
    http://www.websitebaker.org/2/about/
  • Add tag where you want to start. eg.
    <h1>About</h1>
  • Add tag where you want to end. eg.
    downloads page</a>.
  • Test to see if the page works

Need to know:
At the moment it is not possible to parse relative links. This can be a problem with the css style and images. I am working on it Smiley

Warning: Stealing is NEVER a good idea. Make sure you are allowed to use a snippit of a page. Sometimes there is copyright on the material.

Languages supported:
  • English
  • Dutch
  • German (by Matthias)

To do:
  • Making it possible to parse $_POST
  • Source field
  • Multi language support
  • Cleaning up code and better design modify page
  • Rewrite relative links from source link
  • Access to remote site with username/password

Changelog: (Thank you Matthias)
  • Replaced all variables $module_directory with hardcoded curl to prevent install errors
  • Made install script mysql5 strict mode compatible
  • Layout changes in the modify window
  • If curl is not loaded only a warning is displayed in modify window
  • Changed language file inclusion
  • Added german language file
  • Made the link of the wrapped page in view.php clickable
  • Code sorting and a few minor fixes in all files
  • Replaced all copyright notices

Update:
April 23 2006: Version .03 released.
April 23 2006: Version .04 released.
April 24 2006: Version .05 released.
April 25 2006: Version .06 released.
April 25 2006: Version .07 released. Thank you Pcwacht!
October 22 2006: Version .08 released. Thank you Matthias!

If you have any suggestions please let me know. If you translated the text file in /languages/ please send it to me so I can include it.

Marco de Moulin
« Last Edit: January 03, 2007, 03:04:58 PM by moulin » Logged
valerie

Offline Offline

Posts: 156


WWW
« Reply #1 on: April 21, 2006, 01:36:01 PM »

Can you help me understand what this does differently from the wrapper?
Logged

http://www.robcomm.net
--editing and proofreading for American English
--print and web publication management
--graphic and web design, including original photography
moulin

Offline Offline

Posts: 32


« Reply #2 on: April 21, 2006, 02:09:58 PM »

With a wrapper you get the full html page from another site. With Curl you can get a part of that page. For example if you want to have the weather forecast from a site you might only want to have 'today's weather' intead of everything. With Curl this is very easy.
Logged
kickarse

Offline Offline

Posts: 64


« Reply #3 on: April 21, 2006, 09:25:57 PM »

Very cool this will be handy!
Logged
kweitzel
Forum administrator
*****
Offline Offline

Posts: 6991


WWW
« Reply #4 on: April 21, 2006, 10:06:58 PM »

Hey ... post that one to "ruebenwurzel" and "woudloper" for testing. Then it can go to the add on repository ...

cheers

Klaus
Logged

WebsiteBaker Org e.V. - for WebsiteBaker

moulin

Offline Offline

Posts: 32


« Reply #5 on: April 23, 2006, 01:41:49 AM »

@Kweitzel:
Yes I will do that when I am fully satisfied with the script. I think I can still make some changes and additions.
Logged
Woudloper
Guest
« Reply #6 on: April 24, 2006, 06:52:52 AM »

Interesting module. Saw that you've made a mistake in the 'install.php'. The query body is wrong! I see a reference to galtitle, but this isn't in your table. This mistake results in an error when searching the website.
Logged
moulin

Offline Offline

Posts: 32


« Reply #7 on: April 24, 2006, 10:15:46 AM »

@Woudloper:
Yep I must have missed that one. I did not try out the search function yet. So thanks!
Logged
moulin

Offline Offline

Posts: 32


« Reply #8 on: May 01, 2006, 07:18:38 PM »

Is there anyone who knows how to include a remote css file? How should this be linked to the main css file of website baker?

Marco
Logged
kweitzel
Forum administrator
*****
Offline Offline

Posts: 6991


WWW
« Reply #9 on: May 01, 2006, 10:36:45 PM »

maybe better start a new topic for that ...

cheers

Klaus
Logged

WebsiteBaker Org e.V. - for WebsiteBaker

virgil

Offline Offline

Posts: 179



« Reply #10 on: May 02, 2006, 10:54:16 PM »

@Klaus

y're right...


@Marco

see answer in new topic...

http://forum.websitebaker.org/index.php/topic,3143.0.html

Cheers
Virgil
Logged

Virgil - the pre-baked-stuff baker -   wink)
moulin

Offline Offline

Posts: 32


« Reply #11 on: August 01, 2006, 01:05:59 PM »

I am updating the script. Are there still special needs? Still looking for translaters. Send me a pm.
Logged
ruebenwurzel
WebsiteBaker Org e.V.

Offline Offline

Posts: 8244



WWW
« Reply #12 on: August 01, 2006, 06:44:34 PM »

Hello,

This modul is based on the simple modul. The simple modul works with including info.php and the variable $module_directory. During install process this variable couldn't be used in some cases. So if you update the script replace this variable with the hardcoded moduldirectory to prevent this nstall issues.

Thanks

Matthias
Logged
diskostu

Offline Offline

Posts: 10


« Reply #13 on: October 21, 2006, 02:07:57 PM »

so if on php server by my host curl is not loaded i can't use this module right?

there's a way to do exactly what your module does without needing curl loaded?
thanks
Logged
ruebenwurzel
WebsiteBaker Org e.V.

Offline Offline

Posts: 8244



WWW
« Reply #14 on: October 21, 2006, 05:04:04 PM »

Hello,

@diskostu
Quote
so if on php server by my host curl is not loaded i can't use this module right?
right

Quote
there's a way to do exactly what your module does without needing curl loaded?
no

@all
Attached you find a bugfixed version 0.8 of this modul:

Changelog:
- replaced all variables $module_directory with hardcoded curl to prevent install errors
- made install script mysql5 strict mode compatible
- layout changes in the modify window
- if curl is not loaded only a warning is displayed in modify window
- changed language file inclusion
- added german language file
- made the link of the wrapped page in view.php clickable
- code sorting and a few minor fixes in all files
- replaced all copyright notices

Hope you like it

Matthias


Logged
kozmoz

Offline Offline

Posts: 34


WWW
« Reply #15 on: October 22, 2006, 02:38:28 PM »

When start tag is empty, I get a PHP waring:


Warning: strpos(): Empty delimiter. in /var/www/wb3/modules/curl/view.php on line 71

Logged
ruebenwurzel
WebsiteBaker Org e.V.

Offline Offline

Posts: 8244



WWW
« Reply #16 on: October 22, 2006, 02:59:02 PM »

Hello,

This modul only works when a start and a end tag is defined. If one fails it is correct that a error message is displayed.

Matthias
Logged
clubvibe

Offline Offline

Posts: 17


« Reply #17 on: May 01, 2007, 02:35:49 PM »

I'm trying to use Curl module but it keeps saying .. "Couldn't connect to http://www.xxxx.xxx" no matter what url i use. I checked if Curl is supported in phpinfo and it is. Anyone an idea howcome it doens't work?

I tried to use it with wb 2.6.3 and a beta of 2.6.6. (also inline wrapper module isn't working)
Logged
moulin

Offline Offline

Posts: 32


« Reply #18 on: May 02, 2007, 12:39:35 AM »

I'm trying to use Curl module but it keeps saying .. "Couldn't connect to http://www.xxxx.xxx" no matter what url i use. I checked if Curl is supported in phpinfo and it is. Anyone an idea howcome it doens't work?
Perhaps your host does not support outgoing Internet traffic from their local host. What version of PHP do you use? And do you run Apache?

I tried to use it with wb 2.6.3 and a beta of 2.6.6. (also inline wrapper module isn't working)
The inline wrapper should download the page in your browser and not on the server side. Do you have more problems with other modules?
Logged
clubvibe

Offline Offline

Posts: 17


« Reply #19 on: May 03, 2007, 10:04:53 AM »

Other modules work just fine. You can find my phpinfo here

I can't understand why both modules won't work. I get an error: (3) in using inline wrapper

edit: i tried an other server (PHPinfo) and everything just worked fine! the thing is i don't understand why it doesn't work on this server.

Want do i (well..my webhoster i mean) have to change in the php config doe i have to change to get it working?
« Last Edit: May 03, 2007, 11:16:02 AM by clubvibe » Logged
ruebenwurzel
WebsiteBaker Org e.V.

Offline Offline

Posts: 8244



WWW
« Reply #20 on: May 03, 2007, 10:42:32 AM »

Hello,

as far as i remeber you can use curl only for static html pages not for pages wich are generated from a script.

The fact that wrapper doesn't work too points more too a server config issue. I see at your phpinfo that you have a server with the newest versions of all software (apache 2.2.4, php 5.2.0), maybe the changes in this software causes that you got errors. Look at the changelog of Apache and PHP for to got more info. WB works best with Apache 2.0.x and Php 4.x. As there are a lot of changes in Apache and PHP in the versions you use it could be that some scripts didn't work anymore.

Matthias
Logged
Attila

Offline Offline

Posts: 150



« Reply #21 on: June 25, 2008, 08:59:47 PM »

I installed this curl module on a WB 2.7 installation. Unfortunally I always get the error message "Page is not available at the moment"...

Any ideas if the module should work with WB 2.7?

-- Dave
Logged
kweitzel
Forum administrator
*****
Offline Offline

Posts: 6991


WWW
« Reply #22 on: June 25, 2008, 09:54:04 PM »

Call the "external page" within WebsiteBaker and then parse the WB page. See: [Edit: Link removed] ... there I did this for a german post.

1) the page called "hidden_include" calls the eternal deeplinked page on a different server
2) the page "cURL Test" then calls the WB Page "hidden_include" and displays a part of it.

cheers

Klaus
« Last Edit: February 12, 2009, 10:55:41 PM by kweitzel » Logged

WebsiteBaker Org e.V. - for WebsiteBaker

moulin

Offline Offline

Posts: 32


« Reply #23 on: June 25, 2008, 10:04:56 PM »

It still works in the last version of websitebaker. Perhaps the server you use is not set up right. External connections can be denied or there is no dns set. You could try to do a curl with an ip-address instead of the url.
Logged
Attila

Offline Offline

Posts: 150



« Reply #24 on: June 26, 2008, 07:46:30 AM »

I already tried to parse a local WB page but the error is always the same...  huh
So it doesn't matter if the parsed page is an external or internal URL.

Are there other things I can test? Maybe something is wrong with curl setup on the server? phpinfo says:
cURL support    enabled
cURL Information    libcurl/7.12.1 OpenSSL/0.9.7a zlib/1.2.1.2 libidn/0.5.6

And here is a screenshot of my page:



-- Dave
Logged
Pages: [1] 2   Go Up
Print
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!