Web development
As mentioned in an earlier posting, I will redesign my website to make it more userfriendly and more easy for me to update. A website that is not updated tend not to be very userfriendly.
As will be very obvious from my questions, webtechnology and databases are not my field of expertise. Just as in music, too many of those who are trying to explain tend to take some basic knowledge for granted, and do not explain the very basic concepts. This is why I am posting these questions, hoping to get some feed back from those of you who know these techonolgies. I need to test if my understanding is correct and get i corrected when it is needed.
I am using three different databases, CATraxx, BookCAT and CatVids to keep track of my CDs, books and videos. So far I have been using them for my own books, CDs and videos. (They are among my favourite programs, highly recommended!) But I want to use the same programs as a kind of "back office organizer" for information I put on the web. There are many reasons for that. One obvious reason is that I have been using these programs for years, I know them and I like them, and I still have not seen anything that come close. They can do more than you ever will try, and they are flexible. I can use Custom fields to register guitar tunings, keys, chord progressions, etc. This is not the kind of information a normal collector will be interested in, and you need a flexible program to be able to include this. I want to use the same program to organize the books I have in my bookshelves, as well as well as the books I put in my "book store". It means that I do not have to learn another program, and it will be easy to make more information available on the net. The same goes for CDs and videos.
These programs are based on MS Access. The hosting service from my ISP is based on Microsoft, with asp and asp.net, and MS SQL database. What I am hoping to do, is to transfer the data from the Access databases to MS SQL. And here comes one of my questions on "what you always wanted to know about databases, but where afraid to ask":
Will it at least to some extent be correct to say that databases have three major components: A user interface used to enter and process the data that goes into the tables, the tables where all the data is stored, and then a component used to extract and present the data. My understanding is that many of the fancy scripts that go into the making of a database is in the "data entry module". The result of this process is a number of tables with "static" data. I am not saying that the tables as such are static, but what is stored in the tables are basically static text and numbers, as well as binary files (pictures, formatted text, etc). There may be parameters defining what kind of data that will be allowed, but the data is not processed. (It is a matter of definition when it comes to data such as dates that are stored as a number, but presented in a date format.) The "Query" or "search" component search, combine and present data from the tables.
My idea is to use my favourite databases to organize the data, export the tables to the SQL server, and then add some procedures to extract the information from the SQL tables and present them on the web. I do not need all the search and presentation facilites in CATraxx, BookCAT and CatVids on the net, and I do not need the data entry and data manipulation facilities. But I have not made up my mind on how I will present this. I will probably start with the same sorting options as today (artist, genre, level etc), with not too detailed information. I will include basic information, internal links to pages on my site that are relevant for their guitar playing (when applicable), and include data on books, CDs, videos, instruments and other equipment. I hope that it will be more flexible and offer more options than what I have today, and that it will open up for further development. An important question for me is if this is a workable solution, and if there are ways to export data tables from Access to MS SQL that are not too complcated. (To export the data in XML format is an option. CATraxx, BookCAT and CatVids have good export procedures for data on books/albums and tracks/content, but unfortunatly there are no procedure for exporting the data on artists/persons. So for at least some of the information, there seem to be no way to avoid exporting Access-tables.)
If I manage to get the data into the SQL tables, the next challenge will be to present the data. As I am thinking now, I will probably go for an XML-based solution, based on XSL-stylesheets. CATraxx, BookCAT and CatVids all have many XSL-templates which I think I can start from. But I need to learn a little more about XSL. I wish I could find a WYSIWYG XSL-editor. When I am working with presentation, I prefer to see what it actually looks like while I am working. So far I have only found text-based editors, and I find it hard to imagine what all these lines of code actually will look like.
The rest of what I am planning seem to be more straightforward, at least there are many products available that promise to do the job. I will add some blogging software, better linking management and maybe a little bit more.
Probably to the disappointment to some of you, I will also develop the advertising a bit more. I am signing up with a few more affiliate programs, and will probably add some software to manage advertising. Geotargeting is the most important feature. I does not make sense to advertise in Europe a supplier that will only deliver in US (like MusiciansFriend), or advertise European services in US. It is the little stream of money generated from sales through the affiliate links cover the expenses related to maintaining the site (as well as giving some pocket money to spend on music ...). But I will only pick products and services that I like and use myself. There will be no online casinos, no "viruses" like adware, no pop-ups, etc. I am buying from Amazon US and Amazon UK, as well as from SheetmusicPlus and MusicRoom. I have been using CATraxx, BookCAT and CatVids for years, and Earope is still my favourite ear training program. I would have been buying from MusiciansFriend if they had shipped to Europe, I am using the Kelkoo service every time I am looking for a flight or any other product or service at a good price, I enjoy music on my iPod, I am writing this on my Dell laptop, etc. These are the kind of products and services you will see advertised on my site. In all advertising there is a kind of endorsment.
If some of you could answer or comment some of the questions raised about webdevelopment, I would appreciate either comments in this blog or an e-mail.
PS:
I sometimes wonder what is the secret of the small Norwegian town Moss. It is a small town 5o km south of Oslo, previously most know for its pulp factory which created a peculiar smell over the town. I found the programs CATraxx, BookCAT and CatVids when surfing the net to find a good program for CD organizing. I tried several, and decided to go for CATraxx. Some time later, I realized that this program is developed by a small company (my understanding is that is is a one man company) in this little town. My favourite reference for web technology is w3school at http://www.w3schools.com/. After having used w3school for a while, I realized that this service is run by another small company in the same little town. It is also the home of Helly Hansen, one of the few Norwegian brand names known globally in the consumer market. But despite excellent software and web development and internationally orientet business: They have not been able to make a webpresentation of the town in English. So I include a link to Galleri F15 instead - one of the leadning contemporary art galleries in Norway.
As will be very obvious from my questions, webtechnology and databases are not my field of expertise. Just as in music, too many of those who are trying to explain tend to take some basic knowledge for granted, and do not explain the very basic concepts. This is why I am posting these questions, hoping to get some feed back from those of you who know these techonolgies. I need to test if my understanding is correct and get i corrected when it is needed.
I am using three different databases, CATraxx, BookCAT and CatVids to keep track of my CDs, books and videos. So far I have been using them for my own books, CDs and videos. (They are among my favourite programs, highly recommended!) But I want to use the same programs as a kind of "back office organizer" for information I put on the web. There are many reasons for that. One obvious reason is that I have been using these programs for years, I know them and I like them, and I still have not seen anything that come close. They can do more than you ever will try, and they are flexible. I can use Custom fields to register guitar tunings, keys, chord progressions, etc. This is not the kind of information a normal collector will be interested in, and you need a flexible program to be able to include this. I want to use the same program to organize the books I have in my bookshelves, as well as well as the books I put in my "book store". It means that I do not have to learn another program, and it will be easy to make more information available on the net. The same goes for CDs and videos.
These programs are based on MS Access. The hosting service from my ISP is based on Microsoft, with asp and asp.net, and MS SQL database. What I am hoping to do, is to transfer the data from the Access databases to MS SQL. And here comes one of my questions on "what you always wanted to know about databases, but where afraid to ask":
Will it at least to some extent be correct to say that databases have three major components: A user interface used to enter and process the data that goes into the tables, the tables where all the data is stored, and then a component used to extract and present the data. My understanding is that many of the fancy scripts that go into the making of a database is in the "data entry module". The result of this process is a number of tables with "static" data. I am not saying that the tables as such are static, but what is stored in the tables are basically static text and numbers, as well as binary files (pictures, formatted text, etc). There may be parameters defining what kind of data that will be allowed, but the data is not processed. (It is a matter of definition when it comes to data such as dates that are stored as a number, but presented in a date format.) The "Query" or "search" component search, combine and present data from the tables.
My idea is to use my favourite databases to organize the data, export the tables to the SQL server, and then add some procedures to extract the information from the SQL tables and present them on the web. I do not need all the search and presentation facilites in CATraxx, BookCAT and CatVids on the net, and I do not need the data entry and data manipulation facilities. But I have not made up my mind on how I will present this. I will probably start with the same sorting options as today (artist, genre, level etc), with not too detailed information. I will include basic information, internal links to pages on my site that are relevant for their guitar playing (when applicable), and include data on books, CDs, videos, instruments and other equipment. I hope that it will be more flexible and offer more options than what I have today, and that it will open up for further development. An important question for me is if this is a workable solution, and if there are ways to export data tables from Access to MS SQL that are not too complcated. (To export the data in XML format is an option. CATraxx, BookCAT and CatVids have good export procedures for data on books/albums and tracks/content, but unfortunatly there are no procedure for exporting the data on artists/persons. So for at least some of the information, there seem to be no way to avoid exporting Access-tables.)
If I manage to get the data into the SQL tables, the next challenge will be to present the data. As I am thinking now, I will probably go for an XML-based solution, based on XSL-stylesheets. CATraxx, BookCAT and CatVids all have many XSL-templates which I think I can start from. But I need to learn a little more about XSL. I wish I could find a WYSIWYG XSL-editor. When I am working with presentation, I prefer to see what it actually looks like while I am working. So far I have only found text-based editors, and I find it hard to imagine what all these lines of code actually will look like.
The rest of what I am planning seem to be more straightforward, at least there are many products available that promise to do the job. I will add some blogging software, better linking management and maybe a little bit more.
Probably to the disappointment to some of you, I will also develop the advertising a bit more. I am signing up with a few more affiliate programs, and will probably add some software to manage advertising. Geotargeting is the most important feature. I does not make sense to advertise in Europe a supplier that will only deliver in US (like MusiciansFriend), or advertise European services in US. It is the little stream of money generated from sales through the affiliate links cover the expenses related to maintaining the site (as well as giving some pocket money to spend on music ...). But I will only pick products and services that I like and use myself. There will be no online casinos, no "viruses" like adware, no pop-ups, etc. I am buying from Amazon US and Amazon UK, as well as from SheetmusicPlus and MusicRoom. I have been using CATraxx, BookCAT and CatVids for years, and Earope is still my favourite ear training program. I would have been buying from MusiciansFriend if they had shipped to Europe, I am using the Kelkoo service every time I am looking for a flight or any other product or service at a good price, I enjoy music on my iPod, I am writing this on my Dell laptop, etc. These are the kind of products and services you will see advertised on my site. In all advertising there is a kind of endorsment.
If some of you could answer or comment some of the questions raised about webdevelopment, I would appreciate either comments in this blog or an e-mail.
PS:
I sometimes wonder what is the secret of the small Norwegian town Moss. It is a small town 5o km south of Oslo, previously most know for its pulp factory which created a peculiar smell over the town. I found the programs CATraxx, BookCAT and CatVids when surfing the net to find a good program for CD organizing. I tried several, and decided to go for CATraxx. Some time later, I realized that this program is developed by a small company (my understanding is that is is a one man company) in this little town. My favourite reference for web technology is w3school at http://www.w3schools.com/. After having used w3school for a while, I realized that this service is run by another small company in the same little town. It is also the home of Helly Hansen, one of the few Norwegian brand names known globally in the consumer market. But despite excellent software and web development and internationally orientet business: They have not been able to make a webpresentation of the town in English. So I include a link to Galleri F15 instead - one of the leadning contemporary art galleries in Norway.
posted by Olav Torvund at Wednesday, November 30, 2005
11 Comments:
Hey Olav -
Your site has some good content, but you have waaay too much advertising on it. The content seems to get lost among all the ad banners. You need to strike a better balance between advertising and content. Adding affiliate banners will only turn readers away from your site, not increase clicks.
Dear Olav,
What a wonderful website you have developed! Probably the best I have ever seen.I am a guitar instructor in California.I place a lot of emphasis with students on what I call "Applied Theory for Guitar".
Your material is so complimentary to my ideas on the importance of analysis of what we are doing at any given time while we are playing. I note that you are an
attorney as well. I am not an attorney-but my wife is. I showed her your website and she was jealouis that you have mastered another area besides law!
Kindest regards.
Wayne Lazar
waynelazar@comcast.net
Hello Olav,
Thanks for your site. It helped me alot learning about music theory in general.
To your technical questions:
To migrate a database you would have to export the database schema from the existing database, import it into the new database and then export and import the data itself.
The database schema is the information about the structure of the database. Without it, you would just have a large heap of bits.
If parts of the application logic are done by the database software (and not by the application itself), then you would have to migrate that logic, too. This is not necessary if you plan for a read-only database, but you would need to synchronize the two on a regular basis.
The XML way is an alternative to export the data and the schema in one go. Of course, the import side has to "understand" the XML format.
Or do you plan to use XML for the generation of the web pages ?
Olav,
I am a guitar novice and have learned so much from your site. I thank you very much for the effort you put into your site. I am an accomplished pianist and am now trying to learn guitar. I know some of the theory stuff, but some of the theory is different and getting my fingers to work on the guitar is like starting over again learning piano.
About your web site. I would be happy to help you and answer questions about the design. I have written the back end code for a few data driven web sites, but my graphical design skills are not so good. I'm more of the technical design person. Anyway, the question about the databases. Why not just use the Access Databases as they are? With Visual Web Developer 2005 Express edition (a free download from Microsoft), you can use c# or Visual Basic to get data from Access or SQL databases. You may want to look into that.
D. Anderson
anderson1420 (at) cableone.net
MS has a piece of software (free I believe) that will convert your Access database(s) to SQL Server. I've never used it so don't know how good it is. As I understand there are some things it would be necessary to do by hand. Depends on the complexity of the database. Just Google for Access, SQL Server, and conversion and you should find it. Oh, and thanx for your site. Best.
Hello Olav,
First, thankyou for sharing your knowledge with the world, I just need three life-times to go through it all..
Now, on websites the only suggestion I can make is to try to use an existing CMS (Content managent system) for your website, a good example of free ones would be Mambo (linux/php based) Dotnetnuke (microsoft .net based ) . There are many others that I haven't mentioned though this information may point you in a useful direction.
Regards, Patrick Saunders
pat_01 at fastmaildotcomdotau
Hello Olav,
First up, your site is great. I don't think the advertising is excessive; it's unobtrusive and doesn't come close to the aggressive ads you find on other sites.I like how you include audio and exercises in addition to the theory and explanations. I get materials from my tutor and other sites, but each set of material gives me a different perspective and more subtle insights into guitar playing, and leads to more 'a-ha' and 'yes!' moments.
Now about your site redesign questions:
As far as databases are concerned, the definition you give of the components of a database are workable, especially with MS Access. But Access is sort of a special case where the data entry component, data retrieval component and data storage component are tightly coupled in the same application. In most "industrial strength" databases, the main component is the database engine, whose job is to manage data structures like tables and indexes, parse and execute SQL querys against the data, plus a myriad of other tasks. The programs which update, retrieve, and present the data are completely separate from the database engine - usually they run on a separate machine across a network and can be written with virtually any language or technology. Access applications also use a database engine - called JET, but since Access is a consumer app, these complexities are smoothed over to make an integrated package that everyone can use easily for their database needs. CATraxx looks like a heavily customized Access application. You can however access the data stored by the program independently from any other program by accessing the .mdb files, but you have to know (or figure out) the CATraxx schema - how and where the app stores its data in the different tables, to be able to export all related data on an album item, for example. And never try to enter data directly into the Access database outside the CATraxx application. Fromm what you've described about what you want to do with your "backoffice" data, I can give you some preliminary recommendations:
1. Export the data you want made public from CATraxx and your other applications. You can make a copy of your CATraxx .mdb database, load it into MS Access, create custom query scripts to pull together all the data you want and then export it in a format like CSV, which is plain text and easily compressible (for fast uploading) and very straightforward to insert into a SQL Server database table.
2. Write a .asp page where you can upload the file and use VBScript/ADO to insert the exported data into your SQL Server database.
3. At this point you have 3 choices:
a. If you're going to use a CMS to manage your entire site then you'll probably have to write or modify a custom addon or module for the CMS to display your data the way you want it.
b. Keep your existing site and write ASP pages to take your exported data out of your SQL Server database and display it the way you want. You can present it with the formatting and graphics just the way you want (something that's not straightforward using the CMS.) The only downside is that you would be duplicating a lot of code that a CMS would have.
c. Export your data as XML and transform the XML data using XSLT to HTML pages. Althought XML is very cool I wouldn't recommend you go this route; it's very difficult to write XSLT to generate complex pages (with frames, paging and on) and you would be stuck with the problem of having to update this large XML file which would be even more error-prone that updating static pages.
If you really want to use XML and XSLT, you can create a template with your desired look and feel using your plain HTML/WYSIWYG editor and then use XSLT to generate a HTML table (for example) to display the dynamic pieces of the page - the Album/artist/track details and so on.
A 4th possibility is to customize and existing CMS to match exactly what you want to do with your site. This isn't as hard as it sounds as there are many, many open-source CMS apps out there. You can check here for "pure" ASP apps, suitable for use with your current hosting:
http://www.google.com/Top/Computers/Programming/Internet/ASP/Applications/Content_Management/Soop_Portal/
but I would recommend starting with something like Plone - http://www.plone.org
because lots of ASP code can get tangled and very hard to manage.
I'll sign off now, because this is a lot of info to digest but I hope it helps. You can always e-mail me at 000neuraln3t-pu000b@yahoo.com,
(remove the zeros to get the actual address)
I could definitely lend a helping hand with web programming in return for all the hard work you put in on this site.
Regards,
Allister.
This is reply to your "primary chords" section. You mentioned your uncertainty about the origins or the term "dominant". Check it out. "Dominant" means 1. Exercising the most influence or control. (www.dictionary.com) In latin (condensed): "domin.ari" -
domino, dominare, dominavi, dominatus V
be master/despot/in control, rule over, exercise sovereignity; rule/dominate;
"Sub" roughly translates as "below"; it seems relevant that the "subdominant" is just below the "dominant". (see dominus - owner, lord, master; the Lord; title for ecclesiastics/gentlemen).
I think that the latin may help with the theory!
Olav: How about showing a "Circle of Fourths" since a chord progression like Am, Dm7, G7, C would be more easily understood if moving in a clockwise direction.
George
Olav,
It is straightforward to export your Access databases to SQLServer. Using the existing programs to access them is also pretty trivial but would require the developer of the program to connect to your SQLServer database on your host site from within his program - or even to allow you to switch back and forth between access db or SQLServer.
In fact, the developer of Catraxx might be pleased to help you use and/or extend his software in ways that suit you, particularly if you credit him and/or advertise his products on your site. He may get ideas from that which he'll apply to making his product more robust as well.
In fact, he might be willing to make some of his source code, particularly queries, available to you as templates that you can then modify or use as examples as you decide upon new ways to use your data.
Also, once you have your data in SQLServer, you can extract it as xml.
I know I would assist you if you were using my product.
--dan
Check your Fm chord page; it's labelled 'Em'. Your site is terrific! I use it daily. Thanks for your effort.
Tom Bell, Maui, Hi, USA
Post a Comment
<< Home