Wednesday, June 20, 2007

Initial Release of my web tools

I placed the Erlang-based web tools that I am working on for the development of some of my own applications online. They are very rough, in that they are a big mix of experimentation and trying to get something working to write some actual applications. So, the documentation is weak, and there are still certain conventions and features that I need to work out that are not particularly complete yet. (So it is version 0.0.1 :)) I am not really concerning myself over making the system polished; what I wrote were just things that I found myself comfortable to work with.

The two tools are (with the most dullest of names):

1. Weberl is a request handler that uses Yaws. The difference between this and ErlyWeb is of its heavy dependence on OTP for configuration, and all that OTP provides and that it uses behaviours for controller modules. It uses SGTE as a templating tool.

2. Formtool is two things: a special form processor for handling basic forms and their validation, as well as a form generator given a specific form layout. (I wrote a draft of an academic paper on this project a month or two ago on this, as a joke. I was going to put that in the archive for this project, but I just got too embarrassed to release it at the last moment. :))

There are other things I wrote that might find their way into the above, but they are not for public consumption now. :)

Links:
Weberl
Formtool

They are both licensed under the BSD license.

If you see anything interesting in this stuff, and have a million questions for me, then you can email me at brian@qinternet.com.

Wednesday, May 02, 2007

Framework Experiment

Today I was experimenting with a few ideas for an Erlang web framework.

I wanted to utilize OTP like features more, like using a .app file, making the application a server, writing controllers using the behavior feature. I also want to see where I can use the release handling features, since I depend on the .app file for configuration.

It works, sort of. It ain't complete. The server does not work. But, other than that, right now, it is quite simple.

I am not ready to release it, but I put the [incomplete] docs online for now. It is at [http://www.pushon.org/overview.edoc]. If those see this as something interesting to work with, I'll eventually set up a subversion repository.

Sunday, April 15, 2007

Simple Dynamic Record Access, Redux

I sparked a little interest in my post. I thank everyone who has commented on my post, both the comment on Reddit, and the comments made here. Now on my list of things to do is to read deeper about this topic.

I will admit that I did not make the most spectacular of implementations of the idea (with a max of like 3 minutes actually thinking about it, just to get something working, excluding writing the entry :))

I did think I was a bit hasty in saying 'hate' like in the phrase 'I hate records'. I 'wish' they were more dynamic, but I am not weeping over it, since myself, ayrnieu (http://programming.reddit.com/info/1i147/comments) and Philip Robinson (http://chlorophil.blogspot.com/2007/04/dynamic-record-access-functions-with.html) were able to tackle something over the idea. I knew its limitations, so I have tended to use proplists mostly instead, for small lists. It was when I had to consider trying to do dynamic access, I was saying "ohhhhh!"

Eh, anyway, forgive the non-technical response. I am going to do some cutting-and-pasting later, replacing my own with a better implementation. :D

Saturday, April 14, 2007

Simple Dynamic Record Access

There is one thing I hate about Erlang, and it is records. It is not so much that the syntax is annoying (yet I am getting used to it), but you cannot use that syntax for dynamic field access; records are a compile-time feature.

So, without using Yariv's Smerl for something like this, I opted for a simpler solution. Records are tuples after all, and you can get record information in a list. So, you can use these two facts to create a function to get the field you specify. Not anything stellar, but for dynamic record access, it works:


find(Item, List) ->
find_(Item, List, 1).
find_(Item, [], _) -> not_found;
find_(Item, [H|T], Count) ->
case H of
Item ->
Count;
_ ->
find_(Item, T, Count+1)
end.

get_rec_value(Key, Rec, RecordInfo) ->
case find(Key, RecordInfo) of
not_found ->
undefined;
Num ->
element(Num+1, Rec)
end.




I couldn't find something like "lists:find/2", so I implemented my own. But the main function is "get_rec_value/3". So you do this:

1> rd(person, {id, name, email}).
2> get_rec_value(name, #person{name="Brian"}, record_info(fields, person)).
"Brian"

For fun, I also did setting record value.


set_rec_value(Key, Value, Rec, RecordInfo) ->
RecList = tuple_to_list(Rec),
case find(Key, RecordInfo) of
not_found ->
Rec;
Num ->
List1 = lists:sublist(RecList, Num),
List2 = lists:sublist(RecList, Num+2, length(RecList)),
tuple_to_list(List1 ++ [Value] ++ List2)
end.


3> set_rec_value(email, "c", #person{name="Brian", id="1", email="b"}, record_info(fields, person)).

I know this stuff looks really simple, but since I thought this was a bit of a nuisance, the fact that I cannot access records without hardcoding the values, something needed to be written.

Did I miss something though? I'd like to know. Please comment. :)

Wednesday, April 11, 2007

The Case For Erlang For Web Applications

This is the topic I am working on for a paper for my final-year project. The problem with the topic is that it is seriously general, since it can cover so much, like performance and scalability concerns, development concerns, etc. So, while in the midst of preparing to debate this question, I have sat and asked myself "Is there a case to use Erlang for web applications?"

So I am focusing on two areas: "How well can a single Erlang node (the interpreter + the hardware) handle a significant load?" I will admit that I never deployed any web application that is used by more than at most, 10-20 people. There are a lot of concerns that I might be overlooking when considering scalability and performance aspects, so I might be jumping in this naively. However, I am a bit inspired by this one point: "If Erlang can perform well on a single piece of hardware, then can this make scalability potentially easier?" This reminds me of the things I have been reading about Ruby on Rails and the scalability problems that people have been having with that, even though I keep hearing "Well, if you are getting a a lot more hits than your system can handle, then I think you can afford more hardware." Potentially, but it seems very convenient to say when you have to actually deploy a system because your current server setup cannot handle the load.

So, bandwidth excluded, what if my small setup of Rails servers cannot take a current spike because of some Slashdot effect occurring? Or what if I have my Web2.0ish application, dinky and small as it is, become the talk on people's lips, causing a spike in load? (I think that would be wonderful, though ;) )

The fact is, I don't know what would happen. But, at the same time, I am not denying the efficacy of well-designed setups that are running Rails applications. But I am assuming something: if they say that Erlang can potentially handle a lot of load, very well, and if it was designed with this in mind, with a very practical and complex application in mind, can it not help when running my not-so-complicated, dinky little web site? Okay, I might get 10 hits on it every week, but even if I only getting 10 hits, Erlang is not really a sledgehammer to drive in a finish nail. That's where Java would come in.

The fact is, with a little effort, one will find doing web development in Erlang to be easy. Okay, it is not up to the level of Rails in terms of features. However, even regardless of the performance and scalability features that are in Erlang, it is showing me, incrementally, to be a very simple but very powerful language framework to get things done.

I said in an earlier post that I am finding it easier to debug certain things because there is no indeterminacy in variables. Erlang is also dynamically typed, so it, initially was easy to pick and learn, and apply my existing understanding of recursion and of higher-order functions to this language.

So, what I am trying to stress here is that, I think, for what I know, as compared to what other people have been saying, I think that using Erlang, with a platform that has been proven for the types of things that it was designed to do, along with its ease of programming, can be said to be a good platform for web application development. It might not be *perfect* (like the things people complain about in terms of string handling), but these things do not outweigh the positives.

This, is, of course, all a crazy assumption, still. I still have yet to get my tiny web app running to actually run tests on it.

The thing that I am worried about though is how to say all the stuff I said in the typical dry academic tone that I will be compelled to use for my final paper. Ugh. But one reason I wrote this is to figure out how to do just that. The thing also is that all the assumptions I have made here are still willingly open to skepticism, and I open and willing to challenge those assumptions.

I just hope that anyone who is reading this might not immediately jump to the conclusion that I am nuts or I don't know what I am doing; my daily blog readings on Reddit and other places give off this sense, particularly if the object of debate is an article that seems too sure of itself.

Saturday, April 07, 2007

Some Observations

I have been working on a number of Erlang-based projects this week; it's spring break! Most of them are just simple libraries. I would think that my code is not the greatest, but I did notice a few things.

1. Having variables not change state does help. Coupling that with unit testing, it makes debugging quite straightforward. The fact is, is that when you focus using functional techniques, by that virtue alone, you remove indeterminacy in what values variables are. Imperative programming did tend to make me a sloppier programmer, I think, and for this, I always felt a bit uncomfortable with the type of code I was writing. In Erlang, most errors that I am making are mostly silly pattern matching errors. I will admit, in this regard, just in terms of the variable issue, I feel more confident with the code that I am writing.

2. I faced some challenges when I was dealing with looping over some data structure that seemed perfect for a regular loop. In these cases, given that I forced myself to pose the question in a functional way, the resulting code was more concise in the end.

3. Besides minor problems that would come up, I have been consistently producing semi-complicated functions that work correctly the first time. I have not documented specifically when this has occurred. All I remember is that it happened a number of times and I was pleasantly surprised. It is a pleasant surprise because it never seemed to happen that way when working in an imperative language. In functional programming, with Erlang, there are cases where I consider what the problem is, write a little documentation to describe it, then just write it (I have not committed myself to TDD yet). Thinking functionally gives me a chance to look at the problem on a very high level, like, in terms of how the data structure is structured, the types of transformations that are applied to the structure, and what the results of functions over the data structure are. With those in mind, producing good-working functions tend to be the result.

4. I will admit that I have not worked much concurrency with Erlang. Most of the work I have done has been using sequential Erlang. Concurrency is a little harder, but the times I have actually worked with it were times when I just sat down and coded something, not exactly knowing what I was coding. I wrote a short program that opens a large number of processes and then each process sends a message to each other process. There was some weird things that were occurring with it, but it works, and not being a multi-threading wizard by a long shot, I think this was really really nice.

5. Pattern matching is probably one of the most useful language features next to variables not changing values (of course though, the two ideas are connected, since variables are patterns.) The type of things that I am doing with pattern matching is just utterly amazing; I use it in every program I write. The power lies in how I am able to describe large ideas through simple functions, by a matter of how the particular function accepts its input. In one tool I am writing (the one I mentioned in the prior post), I can easily categorize a cross-section of features for different parts of the execution of the tool using patterns; the approach is concise and is very clear to me. Here is an example:


when_valid(login, Form, Args) ->
Vals = erlyform_maker:form_values(Form),
Results = yak:login_user(proplists:get_value(username, Vals),
proplists:get_value(p1, Vals)),
case Results of
{error, user_not_found} ->
{failure, "Username or password was incorrect. Try again."};
{atomic, SessionId} ->
yaws_api:setcookie("yak", SessionId}
end;



when_valid(signup, Form, Args) ->
Vals = erlyform_maker:form_values(Form),
Results = yak:create_user(proplists:get_value(username, Vals),
proplists:get_value(p1, Vals)),
case Results of
{error, username_already_exists} ->
{failure, "The username you selected already exists."};
_ ->
ok
end.


In the code above, I handle a case where web form data is valid. I have two such forms in this case, and they are both related to authentication. My tool goes ahead and validates the data that comes in and is able to call the right function above. I can't explain how neat this is. I wish Python had something like this.

(Let me explain a design goal in this. If a user submits a form on the web, it comes to my server and my tool checks to see if the form is valid. If so, it calls the "when_valid" function. If there is no "when_valid" function, it calls a default version. The "when_valid" function just returns data. This is opposed to a normal ErlyWeb controller where you return an ErlyWeb tuple. I separated that out, since I never liked framework-level code to be mixed in such a way with this type of application logic.)

6. Not worrying about OOP has helped me immensely. When I stopped thinking about OOPish things, which is one reason why I started using functional programming more, it has helped me focus on my problems more. There are a number of things that have started to go through my head on this issue. Of course, it is a bit personalized. One of the main issues is that, whenever I am working in an OOP language, I feel obligated to do things in an OOPish way and not in a way that solves the problem most simply, as well as cleanly. This is merely my way of wanting to use the best practices of the language. To me though, the best practices of a given OOP language are not really the best practices to me.

The confirmation of my incremental success that I stated earlier lies in the absence of OOP. Maybe for big enterprise-y stuff it works nice. Maybe it is nice to know what a particular piece of data is, in the context of a software system. Maybe something like data encapsulation on objects really does wonders for ensuring some form of safety of implementation in big software systems. But, for the first, I am not working on enterprise-y stuff. For the second, if I need to say what something is, I put it in a tuple, and document that, for example, the data that is being returned is a tuple that begins with a specific atom. And, for the third, if I need data encapsulation, I always got the support of Erlang's module system.

I could go on what *I* do not need in OOP. This is not the point. I have come to the opinion now if I am worrying myself over side concerns to suit the best practices of a language, maybe I should try another language. This is, at least, for cases where I have complete control over language choice.

I do worry over Erlang standards. I lay out all my projects the OTP way. I try to focus on writing code with the principle of "just let it crash if it fails." But see, these things don't get in my way.


Anyway, I went on too long. So, I'll shush now.

Saturday, March 24, 2007

Web Forms With Erlang

I started a project that I eluded to in my previous entry. I only wrote the specification and started on some code. so it is far from complete. This is what I wrote so far as a "goal" statement:

A mini-framework on top of ErlyWeb to:

1. Generalize a common set of patterns when related to web form development.
2. Siimplify the creation of forms for instant gratification.

This is accomplished by:

1. A general web form behaviour in Erlang, which handles generating new and edit forms as well as deleting and retrieving data for results or details.

2. A specific way of creating multiple amount of forms in a unit.

3. A way to quickly generate forms, which can be played with immediately.

4. An HTML form generator that takes form specificiations an can genereate simple forms as something to start off with before a better design is created.

5. The allowance of the system to easily adapt to new HTML designs and code.


The idea is this: I got a form of something - be it data to fill in from one or more database tables, or email to send, or whatever. The implementation makes a common set of assumptions about what is going to happen to the data.

The assumptions are this:

- This something will be created.
- This something will be edited.
- This something will be deleted.
- This something will be retrieved.

What is being modeled is the common expectations of what will be performed:

- Data needs to be validated
- If validations succeed, we need to know what to do next.
- If the validations fail, we need to know what to do next.

The system will have a sensible set of defaults, making it possible to make a form quickly, with the validations, and the form can immediately be tested (without any database code or what not.) When the form is shown to work, the details can be plugged in. (One idea that came to my head was to have the framework make a temporary table in Mnesia, so the retrieving aspect can be quickly tested.)

My idea is to utilize an Erlang behaviour in this case. With a behaviour, the compiler can check if my specific forms are missing functions, and warn the developer of the missing function.

The result is a framework the mixes a bit of simplicity (sort of what you get with ErlyWeb or Rails instant CRUD generators) as well as being general enough to be actually useful outside of just simple CRUD.

In this process, I have started to rewrite my HTML form builder I mentioned in a prior post. There is a lack of generality in it that I need to fix.