Constructing skill trees: Difference between revisions

From formulasearchengine
Jump to navigation Jump to search
← Older edit
en>Truthanado
c/e
en>Monkbot
 
Line 1: Line 1:
{{Orphan|date=July 2012}}
If you are interested in home improvement but don't know where to start, don't worry. There are lots of great resources available no matter what kind of project you are considering. Whether it's fixing a leaky faucet or completely remodeling your home, this article can give you tips to help you on your way.<br><br>Use an old, dirty paint roller to clean the gutters on your house. The old paint roller that you have [http://www.Google.co.uk/search?hl=en&gl=us&tbm=nws&q=deemed+unusable&gs_l=news deemed unusable] will quickly and easily clean your dirty gutters. Attach the roller to an extended handle. Then simply remove debris and leaves by sliding it along the gutter.<br><br>Self adhesive drywall patches are a must-have home improvement. These patches come in many forms ranging from mesh drywall tape to those made of a polymer material. Press them carefully into place using an iron. Be careful not to scorch the patch or the wall surface. Paint right over the patch and you'll never know it's there.<br><br>Make your child a room-sized blackboard! It will provide hours of entertainment and offer interest to practically any room. All you have to do is paint a section of a wall with paint that's made especially for blackboards. If you want, you can even frame it in with molding to give it that professional look.<br><br>Spice up the look of your rooms by using patterned wallpaper on your walls and hanging a conversation piece of artwork. Pick something interesting with many textures or images to look at. Don't be afraid to go bold and interesting to show a sense of personality through your choices. You can even paint something yourself and hang it up to match your wallpaper.<br><br>Get someone knowledgeable to give your existing flooring the once over before you pay out for brand new hardwood flooring. You may have some great, natural hardwood beneath your linoleum or carpet that just needs refinished. You can wind up with nice, high-quality flooring for far less money than purchasing new hardwood.<br><br>An important tip to remember with real estate is that when you are making improvements to your house, you should not date it with fashions that are on their way out. This is important because while something may look nice for the time being you should always be forward thinking, especially if you do plan on selling your house.<br><br>If you are working with a very small (less than eight feet wide) section of kitchen counter space, consider visiting a few local granite dealers in search of remnants. Larger dealers often discard these remnants and only charge a small amount for cutting and installing a small counter top. This is an excellent way to add high style to a small surface, and at a great deal no less.<br><br>While wood fireplaces can keep your house toasty warm, consider other heating sources first. Although it looks great, a lot of the heat escapes through the chimney, so it is not very practical. Fireplaces can also deplete the oxygen level within your home since the fire uses oxygen to keep itself burning.<br><br>Paint worn kitchen floors. If you have a wood floor in your kitchen, it will suffer a lot of abuse over the years. If it gets to the point where sanding and re-staining won't do the job, try painting it. A coat of oil-based outdoor paint is strong enough for everyday wear, and it will give your kitchen a cottage feel, which is very fashionable nowadays.<br><br>If you're doing some home improvement, look into plumbing and rewiring issues first. You generally want to do these prior to other jobs since they require access to the inside of your walls. Once your walls are already open, it is much easier to tackle general [https://www.rebelmouse.com/dizzytailor4957/great-roofing-tips-you-should--772173738.html maintenance] to wiring or plumbing.<br><br>If you are trying to remove wallpaper as part of a home improvement project, especially if you are removing wallpaper with paint on top, consider renting a wallpaper steamer. On plaster walls, a wallpaper steamer can greatly speed up the process of wetting and scraping the wallpaper. However, you should take great care when using a wallpaper steamer on drywall, as the drywall is very easily damaged with all the scraping and steam.<br><br>What you have just learned from the tips above might not be as specific as you need for any one particular project, but you should get a good idea of how you need to prepare for a home improvement project. It's not just something you should leap into. Use what you've learned here to act as your guide.
 
<!-- Please leave this line alone! -->
 
[[Image:Mcar.png|thumb|300px| The Mountain Car Problem]]
 
'''Mountain Car''', a standard testing domain in [[Reinforcement Learning|reinforcement learning]], is a problem in which an under-powered car must drive up a steep hill. Since gravity is stronger than the car's engine, even at full throttle, the car cannot simply accelerate up the steep slope. The car is situated in a valley and must learn to leverage potential energy by driving up the opposite hill before the car is able to make it to the goal at the top of the rightmost hill. The domain has been used as a [[test bed]] in various [[Reinforcement Learning|reinforcement learning]] papers.
 
==Introduction==
The mountain car problem, although fairly simple, is commonly applied because it requires a reinforcement learning agent to learning on two continuous variables, position and velocity. For any given state (position and velocity) of the car, the agent is given the possibility of driving left, driving right, or not using the engine at all. In the standard version of the problem, the agent receives a negative reward at every time step when the goal is not reached; the agent has no information about the goal until an initial success.
 
==History==
The mountain car problem appeared first in Andrew Moore's PhD Thesis (1990).<ref>[Moore, 1990] A. Moore, Efficient Memory-Based Learning for Robot Control, PhD thesis, University of Cambridge, November 1990.</ref> It was later more strictly defined in Singh and Sutton's Reinforcement Leaning paper with [[Eligibility Traces|eligibility traces]].<ref>[Singh and Sutton, 1996] Singh, S.P. and Sutton, R.S. (1996) Reinforcement learning with replacing eligibility traces. Machine Learning 22(1/2/3):123-158.</ref> The problem became more widely studied when Sutton and Barto added it to their book Reinforcement Learning: An Introduction (1998).<ref>[Sutton and Barto, 1998] Reinforcement Learning: An Introduction. Richard S. Sutton and Andrew G. Barto. A Bradford Book. The MIT Press Cambridge, Massachusetts London, England, 1998</ref> Throughout the years many versions of the problem have been used, such as those which modify the [[reward function]], termination condition, and/or the [[start state]].
 
==Techniques Used to Solve Mountain Car==
[[Q-learning]] and similar techniques for mapping discrete states to discrete actions need to be extended to be able to deal with the continuous state space of the problem. Approaches often fall into one of two categories, state space [[discretization]] or [[function approximation]].
 
===Discretization===
 
In this approach, two continuous state variables are pushed into discrete states by bucketing each continuous variable into multiple discrete states. This approach works with properly tuned parameters but a disadvantage is information gathered from one state is not used to evaluate another state. [[Tile Coding|Tile coding]] can be used to improve discretization and involves continuous variables mapping into sets of buckets offset from one another. Each step of training has a wider impact on the value function approximation because when the offset grids are summed, the information is diffused.<ref>http://webdocs.cs.ualberta.ca/~sutton/book/8/node6.html#SECTION00132000000000000000</ref>
 
===Function Approximation===
 
Function approximation is another way to solve the mountain car. By choosing a set of basis functions beforehand, or by generating them as the car drives, the agent can approximate the value function at each state. Unlike the step-wise version of the value function created with discretization, function approximation can more cleanly estimate the true smooth function of the mountain car domain.<ref>http://webdocs.cs.ualberta.ca/~sutton/book/8/node9.html#SECTION00140000000000000000</ref>
 
===Traces===
 
An interesting aspect of the problem involves the delay of actual reward. The agent isn't able to learn about the goal until a successful completion. Given a naive approach without traces, for each trial the car can only backup the reward of the goal slightly. This is a problem for naive discretization because each discrete state will only be backup once, taking a larger number of episodes to learn the problem. To alleviate this problem, traces will automatically backup the reward given to states before dramatically increasing the speed of learning.
 
==Technical Details==
The mountain car problem has undergone many iterations. This section will focus on the standard well defined version from Sutton (2008).<ref>[Sutton, 2008] Mountain Car Software. Richard s. Sutton. http://www.cs.ualberta.ca/~sutton/MountainCar/MountainCar.html</ref>
 
===State Variables===
 
Two dimensional continuous state space.
 
<math>Velocity = (-0.07,0.07)</math>
 
<math>Position = (-1.2,0.6)</math>
 
===Actions===
 
One dimensional discrete action space.
 
<math>motor = (left, neutral, right)</math>
 
===Reward===
 
For every time step:
 
<math>reward = -1</math>
 
===Update Function===
 
For every time step:
 
<math>Action = [-1,0,1]</math>
 
<math>Velocity = Velocity + (Action) *0.001+\cos(3*Position)*(-0.0025)</math>
 
<math>Position = Position + Velocity</math>
 
===Starting Condition===
 
Optionally, many implementations include randomness in both parameters to show better generalized learning.
 
<math>Position = -0.5</math>
 
<math>Velocity = 0.0</math>
 
===Termination Condition===
 
End the simulation when:
 
<math>Position \ge 0.6</math>
 
==Variations==
There are many versions of the mountain car which deviate in different ways from the standard model. Variables that vary include but are not limited to changing the constants (gravity and steepness) of the problem so specific tuning for specific policies become irrelevant and altering the reward function to affect the agent's ability to learn in a different manner. An example is changing the reward to be equal to the distance from the goal, or changing the reward to zero everywhere and one at the goal. Additionally we can use a 3D mountain car with a 4D continuous state space.<ref>http://library.rl-community.org/wiki/Mountain_Car_3D_(CPP)</ref>
 
== References ==
{{Reflist}}
 
==Implementations==
* [http://www.cs.ualberta.ca/~sutton/MountainCar/MountainCar.html C++ Mountain Car Software. Richard s. Sutton. (2008)]
* [http://library.rl-community.org/wiki/Mountain_Car_(Java) Java Mountain Car with support for RL Glue]
 
== Further reading ==
* {{cite paper | id = {{citeseerx|10.1.1.51.4764}} | title = Mountain Car with Sparse Coarse Coding }}
* [http://www-all.cs.umass.edu/pubs/1995_96/singh_s_ML96.pdf Mountain Car with Replacing Eligibility Traces]
* {{cite paper | id = {{citeseerx|10.1.1.97.9314}} | title = More discussion on Continuous State Spaces }}
* [http://www.mendeley.com/research/reinforcement-learning-with-gaussian-processes/ Gaussian Processes with Mountain Car]
<!--- Categories --->
 
[[Category:Articles created via the Article Wizard]]
[[Category:Machine learning]]

Latest revision as of 15:56, 28 July 2014

If you are interested in home improvement but don't know where to start, don't worry. There are lots of great resources available no matter what kind of project you are considering. Whether it's fixing a leaky faucet or completely remodeling your home, this article can give you tips to help you on your way.

Use an old, dirty paint roller to clean the gutters on your house. The old paint roller that you have deemed unusable will quickly and easily clean your dirty gutters. Attach the roller to an extended handle. Then simply remove debris and leaves by sliding it along the gutter.

Self adhesive drywall patches are a must-have home improvement. These patches come in many forms ranging from mesh drywall tape to those made of a polymer material. Press them carefully into place using an iron. Be careful not to scorch the patch or the wall surface. Paint right over the patch and you'll never know it's there.

Make your child a room-sized blackboard! It will provide hours of entertainment and offer interest to practically any room. All you have to do is paint a section of a wall with paint that's made especially for blackboards. If you want, you can even frame it in with molding to give it that professional look.

Spice up the look of your rooms by using patterned wallpaper on your walls and hanging a conversation piece of artwork. Pick something interesting with many textures or images to look at. Don't be afraid to go bold and interesting to show a sense of personality through your choices. You can even paint something yourself and hang it up to match your wallpaper.

Get someone knowledgeable to give your existing flooring the once over before you pay out for brand new hardwood flooring. You may have some great, natural hardwood beneath your linoleum or carpet that just needs refinished. You can wind up with nice, high-quality flooring for far less money than purchasing new hardwood.

An important tip to remember with real estate is that when you are making improvements to your house, you should not date it with fashions that are on their way out. This is important because while something may look nice for the time being you should always be forward thinking, especially if you do plan on selling your house.

If you are working with a very small (less than eight feet wide) section of kitchen counter space, consider visiting a few local granite dealers in search of remnants. Larger dealers often discard these remnants and only charge a small amount for cutting and installing a small counter top. This is an excellent way to add high style to a small surface, and at a great deal no less.

While wood fireplaces can keep your house toasty warm, consider other heating sources first. Although it looks great, a lot of the heat escapes through the chimney, so it is not very practical. Fireplaces can also deplete the oxygen level within your home since the fire uses oxygen to keep itself burning.

Paint worn kitchen floors. If you have a wood floor in your kitchen, it will suffer a lot of abuse over the years. If it gets to the point where sanding and re-staining won't do the job, try painting it. A coat of oil-based outdoor paint is strong enough for everyday wear, and it will give your kitchen a cottage feel, which is very fashionable nowadays.

If you're doing some home improvement, look into plumbing and rewiring issues first. You generally want to do these prior to other jobs since they require access to the inside of your walls. Once your walls are already open, it is much easier to tackle general maintenance to wiring or plumbing.

If you are trying to remove wallpaper as part of a home improvement project, especially if you are removing wallpaper with paint on top, consider renting a wallpaper steamer. On plaster walls, a wallpaper steamer can greatly speed up the process of wetting and scraping the wallpaper. However, you should take great care when using a wallpaper steamer on drywall, as the drywall is very easily damaged with all the scraping and steam.

What you have just learned from the tips above might not be as specific as you need for any one particular project, but you should get a good idea of how you need to prepare for a home improvement project. It's not just something you should leap into. Use what you've learned here to act as your guide.

Retrieved from "https://en.formulasearchengine.com/index.php?title=Constructing_skill_trees&oldid=269514"