Epiphany Extension Finsished , Call For Test

I have finished the epiphany extension for beagle. It is written in python now.
You can follow the instruction in wiki to install and test.
What the extension does:

• auto index (regular expression based filter)
• on-demand index (current page or linked page/file)
• prompt for keyword when on-demand index
  
• i18n support (using gettext)
  

There are some problems not solved yet.

• Currently, there is no UI to edit config , you have to edit config file by hand . But the format is simple , you can have a look at this.
• It still have some problems in content-type detective. E
  ven for loaded pages , epiphany only tell us the page is "HTML" ,"XML","IMAGE", or "OTHER". It's hard to decide the correct content type.
• UI is simple . All pure-text menu items . I should use some icons.

index firefox bookmarks

A few days ago,Debajyoti suggest me to add bookmark index support in firefox extension.
And now the firefox extension can inex the bookmarks.(Just check out the latest code)
FF have many kind of bookmarks. we only index the ordinary ones.
(Livebookmark is not indexed)
What will be indexed (for a bookmark):

• the name of bookmark (it is the page title by default)
• the url
  
• shortcut url (or keywords) if any
  
• description if any
• the path . The name of parent folder and parent of parent folder,........
• and we use the hit type "firefoxbookmark"
  

How?
We just using deep first traverse on the bookmark tree ( in fact it is RDF data source). check every bookmark and index it if it is modified(or added) since last index. And in the end we update the last-index-date .

When?

• every time when menu item "bookmarks->index the modified bookmarks" clicked
  
• when the browser window is closed. (you can turn it off)
• It is advised that you do an initial index by hand (click the menu item)after you install the extension .
  

Known issues

• Every time FF 's livemark is updated , the modified time is changed.
  But it doesn't change any thing we indexed.
  
• It would be better if you index them when firefox exits instead of window close. But the environment we need no longer exists when FF exits.
  

Epipahny extension for beagle

This extension is in python now.

Install

1. download the files from http://browser-extension-for-beagle.googlecode.com/svn/trunk/py-epiphany-extension/
2. put them in ~/.gnome2/epiphany/extensions/
3. create beagleIndexer.conf in the folder. ( beagleIndexer.conf.sample is a good example)

Usage

• index current page by menu item Tools->beagle->index this page
• toggle auto index by menu item Tools->beagle->auto index

Configure File

The config file is a python file itself. It's name must be "beagleIndexer.conf" and located in ~/.gnome2/epiphany/extensions/

table of configure items

name	possible value	description
auto_index	True / False	auto index when page is loaded or not
default_index	True / False	index the page when the url did not match any rule in blacklist or whitelist or not
white_list_first	True / False	whether to index the page when the url matches both rules in blacklist or whitelist or not
index_https	True / False	index https page or not
black_list	a python list of regular expressions	black list
white_list	a python list of regular expressions	white list

You can find a sample config file here.

The implement of "Index This Link"

The browser element : A frame which is expected to contain a view of a Web document.
(From http://www.xulplanet.com/references/elemref/ref_browser.html)
In firefox extension we usually use this element to load an document in the background.
I implement "Index This Link" using browser element.
How ?

1. When the menu item is clicked , we
   
   • get the link URL
     
   • open a new window chrome://newbeagle/content/indexLink.xul , and pass URL and current window element as arguments
2. When chrome://newbeagle/content/indexLink.xul is loaded.
   
   • init beagleInvisibleBrowser with opener. (the window which this window)
   • load the passed URL
3. once the URL is loaded.
   
   • call winodw.opener.beagle.indexIt with the load document .

In this way, the linked document is indexed just like I loaded it in a new tab , and then "index this page ".
Another problem , why open a new window ?

• We can't put a browser element in the status bar / toolbar or any other place using overlay. So We have put open a new window , and put a browser element in it . (not visible,of course).
  
• We should let user know that we are downloading something. And also it may take some time , We must allow the user to see the process and cancel it .

Coding time is coming!

Tomorrow is May, 28th. Coding time is coming.
What I have already done:

• Preference Window design and implement
• exclude / include rule design (domain,wild card and regular expression)
  
• preference save / load implement ( using json format)

What to do next (in next week):

• Other UI design and implement (context menu,toolbar/status bar icon)
• index html pages
• make it runable (pack it as XPI)

The preferences window

The preferences dialog.



And the add pattern dialog

Some graphs

Not UML , not MindMap. Just some graphs.

exclude or include ?

Exclude or include ?
That is BlackList or WhiteList ?

• Only BlackList.
  In this way we assume that the user want to index most of the pages.
• Only WhiteList.
  In this way we assume that the user only want to index only a few pages.
• Both BlackList and WhiteList
  We assume nothing in this way.
  But there is another problem:
  If an URL matches some rule both in BlackList and WhiteList.
  How to handle it ?BlackList first or WhiteList first ?
• about UI
  two tabs (one for WhiteList, one for BlackList) or
  one tab for both WhiteList and BlackList
• save /load
  Save the WhiltList and BlackList to an XML file.
  Load the list from the saved file.
  Maybe I could make it possilbe to load the file in Epiphany Extension.

The Old TODO List

I'm starting reading the code of old Firefox Extension For Beagle.
I found a TODO List in readme.

- make install rule for installing to global extensions
directory (can be done without running mozilla?)
I don't think that is a good idea.
- need better icons
- only index content on initial visit to a page
- send HTTP headers to beagle-index-url?
Is it needed? HTTP headers don't contain much information about the content.
- remove context menu and preferences?
Create a new UI
- improve warnings
- call beagle-index-url with page title and mime-type
Good Idea, same for non-html files
- watch for beagle-index-url error and clean up tempfile
Is it possible ?
- save tempfile in safer place and with better name
Good Idea .
- include install.js for older versions of Firefox (<0.9)
- screenshot webpage
??

I'm accepted

I'm accepted.
http://code.google.com/soc/beagle/appinfo.html?csaid=C3888F1BDF9F1276
This summer I will work for beagle .
My work is rewriting the browser extensions for beagle.
Thanks beagle . Thanks google.

Browser Extension Rewrite for beagle

Project Goal:
Rewrite the Firefox Extension and Epiphany extension for Beagle.
including interface ,jslib (for firefox extension),etc.
And add some new feature to the extension.

1.Persist the firefox extensions enable/disable settings (bug 315610)
2.adding more powerful filters(using regular expression/or wildcards) to exclude/include the URLs
3.(optional) introduce an interface(for search) for beagle in the browser(may be only for firefox)

Project Details:
As mentioned in http://beagle-project.org/Summer_Of_Code_2007 , The Browser Extensions for Beagle need a complete rewrite.
I am willing to rewrite the extension.
And I'd like to add some new feathers to them.
1.Persist the firefox extensions enable/disable settings
As described in bug 315610,It is unconvenient to stop the index echo time User open a new window. I'd like to introduce this option to the extension.
2.Currently, the firefox extension only do an basic filter based on host.
I think that is not enough. I'd like to add more power filters. Based on wildcard ,or regular expression . (Something like FoxyProxy's URL template)
3.(optional) introduce an interface(for search) for beagle in the browser(may be only for Firefox)using the command line tool beagle-query.
This feathers may have nothing to do with the original goal of the extension.
But I do think it would be useful.


The above description are almost all for Firefox extension. But the feathers can be add to Epiphany extension too.
In fact the Epiphany extension is really simple now,only less than 500 line codes.
It need much improvement. I didn't say much about it because I have no idea of Epiphany extension develop now. But I'm willing to learn it in the future.


Rough Road Map:
1.(now - 28/5/2007)
discuss with Beagle develop team about my idea.I will get familiar with the related code . And learn some something about Epiphany Extension Development.
2.(1/6/2007 - 31/6/2007)
rewrite the firefox extension
develop and test the first two features for firefox extension.
3.(1/7/2007 - 31/7/2007)
develop and test the Epiphany extension
4.(1/8/2007 - 15/8/2007)
develop and test the third feature for the firefox extension.

About me:
I'm a student from China.
I am studying Computer Science And Engineering in Southeast University in Nanjing,China.
I'm interested in Open source , Linux , Python , PHP , Javascript ,etc.
I have some experiences on Firefox Extension Develop.
(The Firefox Extension I developed -- Image Download https://addons.mozilla.org/firefox/2503)
I have no idea of Epiphany extension develop , But I'd willing to learn about it.
Please visit
http://filia.tao.googlepages.com/english-resume
for more information about me.

My Email : Filia.Tao (AT) gmail.com

Tao Fei's Resume

I place my resume here.
http://filia.tao.googlepages.com/english-resume