Code News

Elixir Metaprogramming Basics

  • Metaprogramming is a powerful, yet pretty complex technique, that means a program can analyze or even modify itself during runtime. Many modern languages support this feature, and Elixir is no exception. 

    With metaprogramming, you may create new complex macros, dynamically define and defer code execution, which allows you to write more concise and powerful code. This is indeed an advanced topic, but hopefully after reading this article you will get a basic understanding of how to get started with metaprogramming in Elixir.

    In this article you will learn:

    • What the abstract syntax tree is and how Elixir code is represented under the hood.
    • What the quote and unquote functions are.
    • What macros are and how to work with them.
    • How to inject values with binding.
    • Why macros are hygienic.

    Before starting, however, let me give you a small piece of advice. Remember Spider Man's uncle said "With great power comes great responsibility"? This can be applied to metaprogramming as well because this is a very powerful feature that allows you to twist and bend code to your will. 

    Still, you must not abuse it, and you should stick to simpler solutions when it is sane and possible. Too much metaprogramming may make your code much harder to understand and maintain, so be careful about it.

    Abstract Syntax Tree and Quote

    The first thing we need to understand is how our Elixir code is actually represented. These representations are often called Abstract Syntax Trees (AST), but the official Elixir guide recommends calling them simply quoted expressions

    It appears that expressions come in the form of tuples with three elements. But how can we prove that? Well, there is a function called quote that returns a representation for some given code. Basically, it makes the code turn into an unevaluated form. For example:

    quote do 1 + 2 end # => {:+, [context: Elixir, import: Kernel], [1, 2]}

    So what's going on here? The tuple returned by the quote function always has the following three elements:

    1. Atom or another tuple with the same representation. In this case, it is an atom :+, meaning we are performing addition. By the way, this form of writing operations should be familiar if you have come from the Ruby world.
    2. Keyword list with metadata. In this example we see that the Kernel module was imported for us automatically.
    3. List of arguments or an atom. In this case, this is a list with the arguments 1 and 2.

    The representation may be much more complex, of course:

    quote do Enum.each([1,2,3], &(IO.puts(&1))) end # => {{:., [], [{:__aliases__, [alias: false], [:Enum]}, :each]}, [], # [[1, 2, 3], # {:&, [], # [{{:., [], [{:__aliases__, [alias: false], [:IO]}, :puts]}, [], # [{:&, [], [1]}]}]}]}

    On the other hand, some literals return themselves when quoted, specifically:

    • atoms
    • integers
    • floats
    • lists
    • strings
    • tuples (but only with two elements!)

    In the next example, we can see that quoting an atom returns this atom back:

    quote do :hi end # => :hi

    Now that we know how the code is represented under the hood, let's proceed to the next section and see what macros are and why quoted expressions are important.

    Macros

    Macros are special forms like functions, but the ones that return quoted code. This code is then placed into the application, and its execution is deferred. What's interesting is that macros also do not evaluate the parameters passed to them—they are represented as quoted expressions as well. Macros can be used to create custom, complex functions used throughout your project. 

    Bear in mind, however, that macros are more complex than regular functions, and the official guide states that they should be used only as the last resort. In other words, if you can employ a function, do not create a macro because this way your code becomes needlessly complex and, effectively, harder to maintain. Still, macros do have their use cases, so let's see how to create one.

    It all starts with the defmacro call (which is actually a macro itself):

    defmodule MyLib do defmacro test(arg) do arg |> IO.inspect end end

    This macro simply accepts an argument and prints it out.

    Also, it's worth mentioning that macros can be private, just like functions. Private macros can be called only from the module where they were defined. To define such a macro, use defmacrop.

    Now let's create a separate module that will be used as our playground:

    defmodule Main do require MyLib def start! do MyLib.test({1,2,3}) end end Main.start!

    When you run this code, {:{}, [line: 11], [1, 2, 3]} will be printed out, which indeed means that the argument has a quoted (unevaluated) form. Before proceeding, however, let me make a small note.

    Require

    Why in the world did we create two separate modules: one to define a macro and another one to run the sample code? It appears that we have to do it this way, because macros are processed before the program is executed. We also must ensure that the defined macro is available in the module, and this is done with the help of require. This function, basically, makes sure that the given module is compiled before the current one.

    You might ask, why can't we get rid of the Main module? Let's try doing this:

    defmodule MyLib do defmacro test(arg) do arg |> IO.inspect end end MyLib.test({1,2,3}) # => ** (UndefinedFunctionError) function MyLib.test/1 is undefined or private. However there is a macro with the same name and arity. Be sure to require MyLib if you intend to invoke this macro # MyLib.test({1, 2, 3}) # (elixir) lib/code.ex:376: Code.require_file/2

    Unfortunately, we get an error saying that the function test cannot be found, though there is a macro with the same name. This happens because the MyLib module is defined in the same scope (and the same file) where we are trying to use it. It may seem a bit strange, but for now just remember that a separate module should be created to avoid such situations.

    Also note that macros cannot be used globally: first you must import or require the corresponding module.

    Macros and Quoted Expressions

    So we know how Elixir expressions are represented internally and what macros are... Now what? Well, now we can utilize this knowledge and see how the quoted code can be evaluated.

    Let's return to our macros. It is important to know that the last expression of any macro is expected to be a quoted code which will be executed and returned automatically when the macro is called. We can rewrite the example from the previous section by moving IO.inspect to the Main module: 

    defmodule MyLib do defmacro test(arg) do arg end end defmodule Main do require MyLib def start! do MyLib.test({1,2,3}) |> IO.inspect end end Main.start! # => {1, 2, 3}

    See what happens? The tuple returned by the macro is not quoted but evaluated! You may try adding two integers:

    MyLib.test(1 + 2) |> IO.inspect # => 3

    Once again, the code was executed, and 3 was returned. We can even try to use the quote function directly, and the last line will still be evaluated:

    defmodule MyLib do defmacro test(arg) do arg |> IO.inspect quote do {1,2,3} end end end # ... def start! do MyLib.test(1 + 2) |> IO.inspect # => {:+, [line: 14], [1, 2]} # {1, 2, 3} end

    The arg was quoted (note, by the way, that we can even see the line number where the macro was called), but the quoted expression with the tuple {1,2,3} was evaluated for us as this is the last line of the macro.

    We may be tempted to try using the arg in a mathematical expression:

    defmacro test(arg) do quote do arg + 1 end end

    But this will raise an error saying that arg does not exist. Why so? This is because arg is literally inserted into the string that we quote. But what we'd like to do instead is evaluate the arg, insert the result into the string, and then perform the quoting. To do this, we will need yet another function called unquote.

    Unquoting the Code

    unquote is a function that injects the result of code evaluation inside the code that will be then quoted. This may sound a bit bizarre, but in reality things are quite simple. Let's tweak the previous code example:

    defmacro test(arg) do quote do unquote(arg) + 1 end end

    Now our program is going to return 4, which is exactly what we wanted! What happens is that the code passed to the unquote function is run only when the quoted code is executed, not when it is initially parsed.

    Let's see a slightly more complex example. Suppose we'd like to create a function which runs some expression if the given string is a palindrome. We could write something like this:

    def if_palindrome_f?(str, expr) do if str == String.reverse(str), do: expr end

    The _f suffix here means that this is a function as later we will create a similar macro. However, if we try to run this function now, the text will be printed out even though the string is not a palindrome:

    def start! do MyLib.if_palindrome_f?("745", IO.puts("yes")) # => "yes" end

    The arguments passed to the function are evaluated before the function is actually called, so we see the "yes" string printed out to the screen. This is indeed not what we want to achieve, so let's try using a macro instead:

    defmacro if_palindrome?(str, expr) do quote do if(unquote(str) == String.reverse( unquote(str) )) do unquote(expr) end end end # ... MyLib.if_palindrome?("745", IO.puts("yes"))

    Here we are quoting the code containing the if condition and use unquote inside to evaluate the values of the arguments when the macro is actually called. In this example, nothing will be printed out to the screen, which is correct!

    Injecting Values With Bindings

    Using unquote is not the only way to inject code into a quoted block. We can also utilize a feature called binding. Actually, this is simply an option passed to the quote function that accepts a keyword list with all the variables that should be unquoted only once.

    To perform binding, pass bind_quoted to the quote function like this:

    quote bind_quoted: [expr: expr] do end

    This can come in handy when you want the expression used in multiple places to be evaluated only once. As demonstrated by this example, we can create a simple macro that outputs a string twice with a delay of two seconds:

    defmodule MyLib do defmacro test(arg) do quote bind_quoted: [arg: arg] do arg |> IO.inspect Process.sleep 2000 arg |> IO.inspect end end end

    Now, if you call it by passing system time, the two lines will have the same result:

    :os.system_time |> MyLib.test # => 1547457831862272 # => 1547457831862272

    This is not the case with unquote, because the argument will be evaluated twice with a small delay, so the results are not the same:

    defmacro test(arg) do quote do unquote(arg) |> IO.inspect Process.sleep(2000) unquote(arg) |> IO.inspect end end # ... def start! do :os.system_time |> MyLib.test # => 1547457934011392 # => 1547457936059392 endConverting Quoted Code

    Sometimes, you may want to understand what your quoted code actually looks like in order to debug it, for example. This can be done by using the to_string function:

    defmacro if_palindrome?(str, expr) do quoted = quote do if(unquote(str) == String.reverse( unquote(str) )) do unquote(expr) end end quoted |> Macro.to_string |> IO.inspect quoted end

    The printed string will be:

    "if(\"745\" == String.reverse(\"745\")) do\n IO.puts(\"yes\")\nend"

    We can see that the given str argument was evaluated, and the result was inserted right into the code. \n here means "new line".

     Also, we can expand the quoted code using expand_once and expand:

    def start! do quoted = quote do MyLib.if_palindrome?("745", IO.puts("yes")) end quoted |> Macro.expand_once(__ENV__) |> IO.inspect end

    Which produces:

    {:if, [context: MyLib, import: Kernel], [{:==, [context: MyLib, import: Kernel], ["745", {{:., [], [{:__aliases__, [alias: false, counter: -576460752303423103], [:String]}, :reverse]}, [], ["745"]}]}, [do: {{:., [], [{:__aliases__, [alias: false, counter: -576460752303423103], [:IO]}, :puts]}, [], ["yes"]}]]}

    Of course, this quoted representation can be turned back to a string:

    quoted |> Macro.expand_once(__ENV__) |> Macro.to_string |> IO.inspect

    We will get the same result as before:

    "if(\"745\" == String.reverse(\"745\")) do\n IO.puts(\"yes\")\nend"

    The expand function is more complex as it tries to expand every macro in a given code:

    quoted |> Macro.expand(__ENV__) |> Macro.to_string |> IO.inspect

    The result will be:

    "case(\"745\" == String.reverse(\"745\")) do\n x when x in [false, nil] ->\n nil\n _ ->\n IO.puts(\"yes\")\nend"

    We see this output because if is actually a macro itself that relies on the case statement, so it gets expanded too.

    In these examples, __ENV__ is a special form that returns environment information like the current module, file, line, variable in the current scope, and imports.

    Macros Are Hygienic

    You may have heard that macros are actually hygienic. What this means is they do not overwrite any variables outside of their scope. To prove it, let's add a sample variable, try changing its value in various places, and then output it:

    defmacro if_palindrome?(str, expr) do other_var = "if_palindrome?" quoted = quote do other_var = "quoted" if(unquote(str) == String.reverse( unquote(str) )) do unquote(expr) end other_var |> IO.inspect end other_var |> IO.inspect quoted end # ... def start! do other_var = "start!" MyLib.if_palindrome?("745", IO.puts("yes")) other_var |> IO.inspect end

    So other_var was given a value inside the start! function, inside the macro, and inside the quote. You will see the following output:

    "if_palindrome?" "quoted" "start!"

    This means that our variables are independent, and we're not introducing any conflicts by using the same name everywhere (though, of course, it would be better to stay away from such an approach). 

    If you really need to change the outside variable from within a macro, you may utilize var! like this:

    defmacro if_palindrome?(str, expr) do quoted = quote do var!(other_var) = "quoted" if(unquote(str) == String.reverse( unquote(str) )) do unquote(expr) end end quoted end # ... def start! do other_var = "start!" MyLib.if_palindrome?("745", IO.puts("yes")) other_var |> IO.inspect # => "quoted" end

    By using var!, we are effectively saying that the given variable should not be hygienized. Be very careful about using this approach, however, as you may lose track of what is being overwritten where.

    Conclusion

    In this article, we have discussed metaprogramming basics in the Elixir language. We have covered the usage of quote, unquote, macros and bindings while seeing some examples and use cases. At this point, you are ready to apply this knowledge in practice and create more concise and powerful programs. Remember, however, that it is usually better to have understandable code than concise code, so do not overuse metaprogramming in your projects.

    If you'd like to learn more about the features I've described, feel free to read the official Getting Started guide about macros, quote and unquote. I really hope this article gave you a nice introduction to metaprogramming in Elixir, which can indeed seem quite complex at first. At any rate, don't be afraid to experiment with these new tools!

    I thank you for staying with me, and see you soon.

    5 days 22 hours ago

NASA Just Proved It Can Navigate Space Using Pulsars. Where to Now?

Ford Is Finally Getting Serious About Making Electric Cars

How to Use the Android ListView Component

  • Introduction

    Lists of related items are needed for almost every app. An example of this is your Gmail app, which displays a scrollable list of messages. To add this functionality to your app, make use of the Android ListView component.

    In this tutorial, you will build an app that uses ListView to display a list of data. By the end, you will have a good understanding of ListView and how to use it in your own apps.

    Application Structure

    We'll be building an app with information about different programming languages. The app will contain three activities. The MainActivity will be the top-level activity. From there, the user will navigate to the LanguageCategoryActivity. This will contain a list of programming languages. The third activity will be the LanguageActivity, which will display details of each programming language. The language data will be held in instances of a  Java class.

    Create the Project

    Create a new Android project named Proglist with the company domain name as code.tutsplus.com. This will make the package name com.tutsplus.code.android.proglist.

    You'll need an empty activity called MainActivity, and the layout should be activity_main. Make sure to uncheck the Backwards Compatibility (AppCompat) checkbox.

    The Language Class

    The Language class will be where the activities get their language data from. Each language will be composed of a name, description, and image resource ID.

    In your Android Studio, switch to the Project view and select the com.tutsplus.code.android.proglist package, which can be found in the app/src/main/java folder. Right click on it, and then go to New > Java Class. The name of the class should be Language, and make sure the package name is com.tutsplus.code.android.proglist. Insert the code below into the file you just created.

    package com.tutsplus.code.android.proglist; public class Language { private String name; private String description; private int imageResourceId; public static final Language[] languages = { new Language("JavaScript", "JavaScript, often abbreviated as JS, is a high-level, " + "dynamic, weakly typed, prototype-based, multi-paradigm, and interpreted " + "programming language.", R.drawable.javascript), new Language("Java", "Java is a general-purpose computer programming language that is " + "concurrent, class-based, object-oriented, and specifically designed to have as" + " few implementation dependencies as possible.", R.drawable.java), new Language("Ruby", "Ruby is a dynamic, reflective, object-oriented, general-purpose " + "programming language. It was designed and developed in the mid-1990s by " + "Yukihiro \"Matz\" Matsumoto in Japan.", R.drawable.ruby) }; private Language(String name, String description, int imageResourceId){ this.name = name; this.description = description; this.imageResourceId = imageResourceId; } public String getName() { return name; } public String getDescription() { return description; } public int getImageResourceId() { return imageResourceId; } public String toString() { return this.name; } }

    In the code above, you can see that each Language has a name, description, and image resource ID. The image resource ID refers to language images which will be added soon. We've also created a static array, languages, with some sample data. 

    We've also created a constructor and getters for the private variables. Also note that the string representation of a Language is its name.

    Drawable Files

    The class you created refers to three image resources, so you need to add those image files to the project. You should have a folder called drawable in the app/src/main/res folder. If the drawable folder is not there, go ahead and create it from Android Studio. You can do this by selecting res, right clicking on it, and then going to New > Android resource directory. Choose a resource type of drawable, name the folder drawable, and click OK.

    Download the files from the tutorial source repo and add each one to the app/src/main/res/drawable folder.

    Main Activity Layout 

    Next, you'll need to add a list view to your layout using the <ListView> element. Then you will add an array of entries to the list view by using an android:entries attribute—this will be set to an array of strings. The array will be displayed as a list of text views. So your main activity layout should look like this:

    <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical" tools:context="com.tutsplus.code.android.proglist.MainActivity"> <ListView android:id="@+id/list_options" android:layout_width="match_parent" android:layout_height="wrap_content" android:entries="@array/options"/> </LinearLayout>

    The values in the list view are given by the options array, which is defined in the strings.xml file like this:

    <resources> <string name="app_name">Proglist</string> <string-array name="options"> <item>Languages</item> <item>Frameworks</item> <item>Tools</item> </string-array> </resources>

    You can find the strings.xml file in the app/src/main/java/res/values folder of your project. This will populate the list view with three values: Languages, Frameworks, and Tools.

    Responding to Clicks

    Items in a list view respond to clicks through an event listener. For this application, you need to create an OnItemClickListener and implement its onItemClick() method. The OnItemClickListener listens for when the items are clicked, and the onItemClick() method lets you determine how the activity should respond to the click.

    In the application, you want to start the LanguageCategoryActivity when the first item on the list is clicked. Here's the code to create the listener.

    AdapterView.OnItemClickListener itemClickListener = new AdapterView.OnItemClickListener() { @Override public void onItemClick(AdapterView<?> parent, View view, int position, long id) { if (position == 0) { Intent intent = new Intent(MainActivity.this, LanguageCategoryActivity.class); startActivity(intent); } } };

    Insert it inside the onCreate method. 

    OnItemClickListener is a nested class within the AdapterView class. The parameters passed to onItemClick give it information about the item that was clicked, such as the item's view and its position.

    After creating the listener, you need to attach it to the list view. This is done using the setOnItemClickListener() method, passing the listener as an argument. This call should be added just below the listener you created, still inside the onCreate method.

    ListView listView = (ListView) findViewById(R.id.list_options); listView.setOnItemClickListener(itemClickListener);

    The above code notifies the listener when an item on the list view is clicked. Without this, the items in your list view would not respond to clicks!

    Create the Language Category Activity

    This activity will list all the programming languages. Create a new activity called LanguageCategoryActivity. Let's start with the layout. Here's how it should look.

    <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical" tools:context="com.tutsplus.code.android.proglist.LanguageCategoryActivity"> <ListView android:id="@+id/list_languages" android:layout_width="match_parent" android:layout_height="wrap_content"/> </LinearLayout>

    You may notice that the layout above is very similar to the one used for the main activity. In this layout, however, the list data is not specified. That's because the data has been programmatically specified in the Language class. For data like this, an adapter is used instead.

    An adapter acts a bridge between the data source and the list view. For this app, you'll be making use of array adapters (just one of the several different kinds of adapter).

    An array adapter binds arrays to views. In this case, you will bind the Language.languages array in the list view using an array adapter. Insert the code below inside the onCreate method of the LanguageCategoryActivity.

    ArrayAdapter<Language> listAdapter = new ArrayAdapter<>(this, android.R.layout .simple_list_item_1, Language.languages); ListView listLanguages = (ListView) findViewById(R.id.list_languages); listLanguages.setAdapter(listAdapter);

    This code starts by initializing the array adapter. To initialize the adapter, the type of data contained in the array is specified. You can then inform the adapter of the current activity, a layout resource that specifies how each item in the array should be displayed, and the array itself.

    The array adapter is then attached to the list view using the ListView.setAdapter() method.

    Adding Intent to the Language Category Activity

    When an item in the LanguageCategoryActivity is clicked, you want to pass control to another activity that displays the information of that language. This new activity should be called LanguageActivity.

    You need to obtain the ID of the clicked item as an extra piece of information, and pass it to LanguageActivity. LanguageActivity will use the ID to obtain the details of the right language.

    The code for the intent should be inserted in LanguageCategoryActivity.

    AdapterView.OnItemClickListener itemClickListener = new AdapterView.OnItemClickListener(){ @Override public void onItemClick(AdapterView<?> parent, View view, int position, long id) { Intent intent = new Intent(LanguageCategoryActivity.this, LanguageActivity.class); intent.putExtra(LanguageActivity.EXTRA_LANGUAGEID, (int) id); startActivity(intent); } };

    Here, you start by obtaining the ID of the clicked item. This will be added to the LanguageActivity when it is created.

    In the onCreate method, you'll also need to assign the listener to the list view like so.

    listLanguages.setOnItemClickListener(itemClickListener);Create the Language Activity

    Create a new activity named LanguageActivity. Replace the contents of the layout with what you have below.

    <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" tools:context="com.tutsplus.code.android.proglist.LanguageActivity"> <ImageView android:id="@+id/photo" android:layout_width="190dp" android:layout_height="190dp" /> <TextView android:id="@+id/name" android:layout_width="wrap_content" android:layout_height="wrap_content" /> <TextView android:id="@+id/description" android:layout_width="match_parent" android:layout_height="wrap_content" /> </LinearLayout>

    This layout contains two text views and an image view. The information to be displayed in those views will be retrieved from the Intent that launches the activity. 

    Update your LanguageActivity class to look like the following:

    package com.tutsplus.code.android.proglist; import android.app.Activity; import android.os.Bundle; import android.widget.ImageView; import android.widget.TextView; public class LanguageActivity extends Activity { //1 public static final String EXTRA_LANGUAGEID = "languageId"; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_language); //2 int languageId = (Integer)getIntent().getExtras().get(EXTRA_LANGUAGEID); Language language = Language.languages[languageId]; //3 TextView name = (TextView) findViewById(R.id.name); name.setText(language.getName()); //4 TextView description = (TextView) findViewById(R.id.description); description.setText(language.getDescription()); //5 ImageView photo = (ImageView) findViewById(R.id.photo); photo.setImageResource(language.getImageResourceId()); photo.setContentDescription(language.getName()); } }

    Here's how it works:

    1. The key that the language id was encoded with in LanguageCategoryActivity is defined. 
    2. When the LanguageActivity is started with the intent from LanguageCategoryActivity, the id of the language is retrieved with the intent. After retrieving the information from the intent, you use the getIntent() method to obtain extra information about the language. languageId is the ID of the language.
    3. Populate the name of the language.
    4. Populate the description of the language.
    5. Populate the language image.

    Now run your app to see what you have built!

    Here is a preview of the app.



    Conclusion

    ListView is a very important and useful component for building Android apps, so understanding how to use it is important in your Android development journey. Now that you know how to use a ListView with static list items and how to use an adapter to wire up dynamic items, there's nothing stopping you! Get out there and build some amazing apps.

    And while you're here, check out some of our other Android app tutorials here on Envato Tuts+!

    6 days 16 hours ago

The Project Veritas Twitter Videos Show the Conservative Backlash Against Moderation

Working with External User Researchers: Part I

  • You’ve got an idea or perhaps some rough sketches, or you have a fully formed product nearing launch. Or maybe you’ve launched it already. Regardless of where you are in the product lifecycle, you know you need to get input from users.

    You have a few sound options to get this input: use a full-time user researcher or contract out the work (or maybe a combination of both). Between the three of us, we’ve run a user research agency, hired external researchers, and worked as freelancers. Through our different perspectives, we hope to provide some helpful considerations.

    Should you hire an external user researcher?

    First things first–in this article, we focus on contracting external user researchers, meaning that a person or team is brought on for the duration of a contract to conduct the research. Here are the most common situations where we find this type of role:

    Organizations without researchers on staff: It would be great if companies validated their work with users during every iteration. But unfortunately, in real-world projects, user research happens at less frequent intervals, meaning there might not be enough work to justify hiring a full-time researcher. For this reason, it sometimes makes sense to use external people as needed.

    Organizations whose research staff is maxed out: In other cases, particularly with large companies, there may already be user researchers on the payroll. Sometimes these researchers are specific to a particular effort, and other times the researchers themselves function as internal consultants, helping out with research across multiple projects. Either way, there is a finite amount of research staff, and sometimes the staff gets overbooked. These companies may then pull in additional contract-based researchers to independently run particular projects or to provide support to full-time researchers.

    Organizations that need special expertise: Even if a company does have user research on staff and those researchers have time, it’s possible that there are specialized kinds of user research for which an external contract-based researcher is brought on. For example, they may want to do research with representative users who regularly use screen readers, so they bring in an accessibility expert who also has user research skills. Or they might need a researcher with special quantitative skills for a certain project.

    Why hire an external researcher vs. other options?

    Designers as researchers: You could hire a full-time designer who also has research skills. But a designer usually won’t have the same level of research expertise as a dedicated researcher. Additionally, they may end up researching their own designs, making it extremely difficult to moderate test sessions without any form of bias.

    Product managers as researchers: While it’s common for enthusiastic product managers to want to conduct their own guerilla user research, this is often a bad idea. Product managers tend to hear feedback that validates their ideas and most often aren’t trained on how to ask non-leading questions.

    Temporary roles: You could also bring on a researcher in a staff augmentation role, meaning someone who works for you full-time for an extended period of time, but who is not considered a full-time employee. This can be a bit harder to justify. For example, there may be legal requirements that you’d have to pass if you directly contract an individual. Or you could find someone through a staffing agency–fewer legal hurdles, but likely far pricier.

    If these options don’t sound like a good fit for your needs, hiring an external user researcher on a project-specific basis could be the best solution for you. They give you exactly what you need without additional commitment or other risks. They may be a freelancer (or a slightly larger microbusiness), or even a team farmed out for a particular project by a consulting firm or agency.

    What kinds of projects would you contract a user researcher for?

    You can reasonably expect that anyone or any company that advertises their skillset as user research likely can do the full scope of qualitative efforts—from usability studies of all kinds, to card sorts, to ethnographic and exploratory work.

    Contracting out quantitative work is a bit riskier. An analogy that comes to mind is using TurboTax to file your taxes. While TurboTax may be just fine for many situations, it’s easy to overlook what you don’t know in terms of more complicated tax regulations, which can quickly get you in trouble. Similarly, with quantitative work, there’s a long list of diverse, specialized quantitative skills (e.g., logs analysis, conjoint, Kano, and multiple regression). Don’t assume someone advertising as a general quantitative user researcher has the exact skills you need.

    Also, for some companies, quantitative work comes with unique user privacy considerations that can require special internal permissions from legal and privacy teams.

    But if the topic of your project is pretty easy to grasp and absorb without needing much specialized technical or organizational insight, hiring an external researcher is generally a great option.

    What are the benefits to hiring an external researcher?

    A new, objective perspective is one major benefit to hiring an external researcher. We all suffer from design fixation and are influenced by organizational politics and perceived or real technical constraints. Hiring an unbiased external researcher can uncover more unexpected issues and opportunities.

    Contracting a researcher can also expand an internal researcher’s ability to influence. Having someone else moderate research studies frees up in-house researchers to be part of the conversations among stakeholders that happen while user interviews are being observed. If they are intuitively aware of an issue or opportunity, they can emphasize their perspective during those critical, decision-making moments that they often miss out on when they moderate studies themselves. In these situations, the in-house team can even design the study plan, draft the discussion guide, and just have the contractor moderate the study. The external researcher may then collaborate with the in-house researcher on the final report.

    More candid and honest feedback can come out of hiring an external researcher. Research participants tend to be more comfortable sharing more critical feedback with someone who doesn’t work for the company whose product is being tested.

    Lastly, if you need access to specialized research equipment or software (for example, proprietary online research tools), it can be easier to get it via an external researcher.

    How do I hire an external user researcher?

    So you’ve decided that you need to bring on an external user researcher to your team. How do you get started?

    Where to find them

    Network: Don’t wait until you need help to start networking and collecting a list of external researchers. Be proactive. Go to UX events in your local region. You’ll meet consultants and freelancers at those events, as well as people who have contracted out research and can make recommendations. You won’t necessarily have the opportunity for deep conversations, but you can continue a discussion over coffee or drinks!

    Referrals: Along those same lines, when you anticipate a need at some point in the future, seek out trusted UX colleagues at your company and elsewhere. Ask them to connect you with people that they may have worked with.

    What about a request for proposal (RFP)?

    Your company may require you to specify your need in the form of an RFP, which is a document that outlines your project needs and specifications, and asks for bids in response.

    An RFP provides these benefits:

    • It keeps the playing field level, and anyone who wants to bid on a project can (in theory).
    • You can be very specific about what you’re looking for, and get bids that can be easy to compare on price.

    On the other hand, an RFP comes with limitations:

    • You may think your requirements were very specific, but respondents may interpret them in different ways. This can result in large quote differences.
    • You may be eliminating smaller players—those freelancers and microbusinesses who may be able to give you the highest level of seniority for the dollar but don’t have the staff to respond to RFPs quickly.
    • You may be forced to be very concrete about your needs when you are not yet sure what you’ll actually need.

    When it comes to RFPs, the most important thing to remember is to clearly and thoroughly specify your needs. Don’t forget to include small but important details that can matter in terms of pricing, such as answers to these questions:

    • Who is responsible for recruitment of research participants?
    • How many participants do you want included?
    • Who will be responsible for distributing participant incentives?
    • Who will be responsible for localizing prototypes?
    • How long will sessions be?
    • Over how many days and locations will they be?
    • What is the format of expected deliverables?
    • Do you want full, transcribed videos, or video clips?

    It’s these details that will ultimately result in receiving informed proposals that are easy to compare.

    Do a little digging on their backgrounds

    Regardless of how you find a potential researcher, make sure you check out their credentials if you haven’t worked with them before.

    At the corporate level, review the company: Google them and make sure that user research seems to be one of their core competencies. The same is true when dealing with a freelancer or microbusiness: Google them and see whether you get research-oriented results, and also check them out on social media.

    Certainly feel free to ask for references if you don’t already have a direct connection, but take them with a grain of salt. Between the self-selecting nature of a reference, and a potential reference just trying to be nice to a friend, these can never be fully trusted.

    One of the strongest indicators of experience and quality work is if a researcher has been hired by the same client for more than one project over time.

    Larger agencies, individual researchers, or something in-between?

    So you’ve got a solid sense of what research you need, and you’ve got several quality options to choose from. But external researchers come in all shapes and sizes, from single freelancers to very large agencies. How do you choose what’s best for your project while still evaluating the external researchers fairly?

    Larger consulting firms and agencies do have some distinct advantages—namely that you’ve got a large company to back up the project. Even if one researcher isn’t available as expected (for example, if the project timeline slips), another can take their place. They also likely have a whole infrastructure for dealing with contracts like yours.

    On the other hand, this larger infrastructure may add extra burden on your side. You may not know who exactly is going to be working on your project, or their level of seniority or experience. Changes in scope will likely be more involved. Larger infrastructure also likely means higher costs.

    Individual (freelance) researchers also have some key advantages. You will likely have more control over contracting requirements. They are also likely to be more flexible—and less costly. In addition, if they were referred to you, you will be working with a specific resource that you can get to know over multiple projects.

    Bringing on individual researchers can incur a little more risk. You will need to make sure that you can properly justify hiring an external researcher instead of an employee. (In the United States, the IRS has a variety of tests to make sure it is OK.) And if your project timeline slips, you run a greater risk of losing the researcher to some other commitment without someone to replace them.

    A small business, a step between an individual researcher and a large firm, has some advantages over hiring an individual. Contracting an established business may involve less red tape, and you will still have the personal touch of knowing exactly who is conducting your research.

    An established business also shows a certain level of commitment, even if it’s one person. For example, a microbusiness could represent a single freelancer, but it could also involve a very small number of employees or established relationships with trusted subcontractors (or both). Whatever the configuration,  don’t expect a business of this size to have the ability to readily respond to RFPs.

    The money question

    Whether you solicit RFPs or get a single bid, price quotes will often differ significantly. User research is not a product but rather a customized and sophisticated effort around your needs. Here are some important things to consider:

    • Price quotes are a reflection of how a project is interpreted. Different researchers are going to interpret your needs in different ways. A good price quote clearly details any assumptions that are going into pricing so you can quickly see if something is misaligned.
    • Research teams are made up of staff with different levels of experience. A quote is going to be a reflection of the overall seniority of the team, their salaries and benefits, the cost of any business resources they use, and a reasonable profit margin for the business.
    • Businesses all want to make a reasonable profit, but approaches to profitability differ. Some organizations may balance having a high volume of work with less profit per project. Other organizations may take more of a boutique approach: more selectivity over projects taken on, with added flexibility to focus on those projects, but also with a higher profit margin.
    • Overbooked businesses provide higher quotes. Some consultants and agencies are in the practice of rarely saying no to a request, even if they are at capacity in terms of their workload. In these instances, it can be a common practice to multiply a quote by as much as three—if you say no, no harm done given they’re at capacity. However, if you say yes, the substantial profit is worth the cost for them to hire additional resources and to work temporarily above capacity in the meantime.

    To determine whether a researcher or research team is right for you, you’ll certainly need to look at the big picture, including pricing, associated assumptions, and the seniority and background of the individuals who are doing the work.

    Remember, it’s always OK to negotiate

    If you have a researcher or research team that you want to work with but their pricing isn’t in line with your budget, let them know. It could be that the quote is just based on faulty assumptions. They may expect you to negotiate and are willing to come down in price; they may also offer alternative, cheaper options with them.

    Next steps

    Hiring an external user researcher typically brings a long list of benefits. But like most relationships, you’ll need to invest time and effort to foster a healthy working dynamic between you, your external user researcher, and your team. Stay tuned for the next installment, where we’ll focus on how to collaborate together.

    6 days 19 hours ago

Now On Nintendo Switch, 'Furi' Embraces The Power of a Good Boss Fight

Building Games With Python 3 and Pygame: Part 2

  • Overview

    This is part two of a five-part series of tutorials about making games with Python 3 and Pygame. In part one, I introduced the series, covered the basics of game programming, introduced Pygame, and examined the game architecture. 

    In this part, we'll look at the TextObject class used to render text on the screen. We'll create the main window, including a background image, and then we'll learn how to draw objects like bricks, the ball, and the paddle. 

    The TextObject Class

    The TextObject class is designed to display text on the screen. The case could be made from a design point of view that it should be a sub-class of GameObject as it is also a visual object and you may want to move it. But I didn't want to introduce deep class hierarchies, when all the text that Breakout displays on the screen stays put. 

    The TextObject creates a font object. It renders the text into a separate text surface that is then blitted (rendered) onto the main surface. An interesting aspect of the TextObject is that it doesn't have any fixed text. Instead, it gets a function called text_func() that is called every time it renders. 

    This allows us to update the display of lives and score in Breakout just by providing a function that returns the current lives and current score, instead of keeping track of which text objects display lives and score and updating their text every time they change. This is a neat trick from functional programming, and for larger games it can help you keep everything nice and tidy. 

    import pygame class TextObject: def __init__(self, x, y, text_func, color, font_name, font_size): self.pos = (x, y) self.text_func = text_func self.color = color self.font = pygame.font.SysFont(font_name, font_size) self.bounds = self.get_surface(text_func()) def draw(self, surface, centralized=False): text_surface, self.bounds = \ self.get_surface(self.text_func()) if centralized: pos = (self.pos[0] - self.bounds.width // 2, self.pos[1]) else: pos = self.pos surface.blit(text_surface, pos) def get_surface(self, text): text_surface = self.font.render(text, False, self.color) return text_surface, text_surface.get_rect() def update(self): passCreating the Main Window

    Pygame games run in windows. You can make them run fullscreen too. Here is how you display an empty Pygame window. You can already see many of the elements I discussed earlier. First, Pygame init() is called, and then the main drawing surface and the clock are created. 

    Next is the main loop, which consistently fills the screen with uniform gray and calls the clock tick() method with the frame rate.

    import pygame pygame.init() screen = pygame.display.set_mode((800, 600)) clock = pygame.time.Clock() while True: screen.fill((192, 192, 192)) pygame.display.update() clock.tick(60)Using a Background Image

    Usually, a uniform color background is not very exciting. Pygame does images very well. For Breakout, I splurged and went for a fancy real space image from NASA. The code is very similar. First, just before the main loop, it loads the background image using the pygame.image.load() function. Then, instead of filling the screen with color, it "blits" (copy the bits) the image to the screen at position (0,0). The effect is that the image is displayed on the screen.

    import pygame pygame.init() screen = pygame.display.set_mode((800, 600)) clock = pygame.time.Clock() background_image = pygame.image.load('images/background.jpg') while True: screen.blit(background_image, (0, 0)) pygame.display.update() clock.tick(60)Drawing Shapes

    Pygame can draw anything. The pygame.draw module has functions for drawing the following shapes:

    • rect 
    • polygon
    • circle
    • ellipse
    • arc
    • line
    • lines
    • anti-aliased line 
    • anti-aliased lines

    In Breakout, all the objects (except the text) are just shapes. Let's look at the draw() method of the various Breakout objects.

    Drawing Bricks

    Bricks are bricks. They are just rectangles. Pygame provides the pygame.draw.rect() function, which takes a surface, a color, and a Rect object (left, top, width and height) and renders a rectangle. If the optional width parameter is greater than zero, it draws the outline. If the width is zero (which is the default), it draws a solid rectangle.

    Note that the Brick class is a subclass of GameObject and gets all its properties, but it also has a color it manages itself (because there may be game objects that don't have a single color). Ignore the special_effect field for now.

    import pygame from game_object import GameObject class Brick(GameObject): def __init__(self, x, y, w, h, color, special_effect=None): GameObject.__init__(self, x, y, w, h) self.color = color self.special_effect = special_effect def draw(self, surface): pygame.draw.rect(surface, self.color, self.bounds)Drawing the Ball

    The ball in Breakout is just a circle. Pygame provides the pygame.draw.circle() function that takes the color, center, radius and the options width parameter that defaults to zero. As with the pygame.draw.rect() function, if the width is zero then a solid circle is drawn. The Ball is also a derived class of GameObject. 

    Since the ball is always moving (unlike the bricks), it also has a speed that is passed on the GameObject base class to be managed. The Ball class has a little twist because its x and y parameters denote its center, while the x and y parameters passed to the GameObject base class are the top-left corner of the bounding box. To convert from center to top-left corner, all it takes is subtracting the radius. 

    import pygame from game_object import GameObject class Ball(GameObject): def __init__(self, x, y, r, color, speed): GameObject.__init__(self, x - r, y - r, r * 2, r * 2, speed) self.radius = r self.diameter = r * 2 self.color = color def draw(self, surface): pygame.draw.circle(surface, self.color, self.center, self.radius)Drawing the Paddle

    The paddle is yet another rectangle that is indeed moving left and right in response to the player's pressing the arrow keys. That means that the position of the paddle may change from one frame to the next, but as far as drawing goes, it is just a rectangle that has to be rendered at the current position, whatever that is. Here is the relevant code:

    import pygame import config as c from game_object import GameObject class Paddle(GameObject): def __init__(self, x, y, w, h, color, offset): GameObject.__init__(self, x, y, w, h) self.color = color self.offset = offset self.moving_left = False self.moving_right = False def draw(self, surface): pygame.draw.rect(surface, self.color, self.bounds)Conclusion

    In this part, you've learned about the TextObject class and how to render text on the screen. You've also got familiar with drawing objects like bricks, the ball, and the paddle.

    In the meantime, remember we have plenty of Python content available for sale and for study in the Envato Market.

    In part three, you'll see how event handling works and how Pygame lets you intercept and react to events like key presses, mouse movement, and mouse clicks. Then, we'll cover gameplay topics like moving the ball, setting the ball's speed, and moving the paddle.  


    6 days 21 hours ago

The Astrophysicist Who Wants to Help Solve Baltimore's Urban Blight

Canny Edge Detector Using Python

  • Edge detection is an essential image analysis technique when someone is interested in recognizing objects by their outlines, and is also considered an essential step in recovering information from images. 

    For instance, important features like lines and curves can be extracted using edge detection, which are then normally used by higher-level computer vision or image processing algorithms. A good edge detection algorithm would highlight the locations of major edges in an image, while at the same time ignoring any false edges caused by noise. 

    But what are edges anyway? Edges are image features that can be used in estimating and analyzing the structure of objects in an image. They represent significant local changes that happened in the image intensity (i.e. pixel value). Edges normally occur on the boundary between two different regions in the image.

    In this tutorial, I'm going to describe the Canny edge detector algorithm, and how we can implement it in Python.

    Canny Edge Detector

    The Canny edge detector algorithm is named after its inventor, John F. Canny, who invented the algorithm in 1986. The Canny edge detector normally takes a grayscale image as input and produces an image showing the location of intensity discontinuities as output (i.e. edges).

    I don't want to go mathematical here, but I will describe what's going on behind the scenes in the Canny edge detector algorithm from a high-level viewpoint.

    The first thing the Canny edge detector does is that it uses Gaussian convolution to smooth the input image and remove noise. A first derivative operator is then applied to the smoothed image in order to highlight those regions of the image with high first spatial derivatives.

    The algorithm then finds both the gradient magnitude and direction by calculating the x-derivative and the y-derivative, especially since knowing the direction of the gradient actually enables us to find the direction of the edges.

    The algorithm then performs what's called non-maximal suppression, where it tracks along the top of the ridges that rise from the edges, and sets those pixels that are not on the ridge top to zero, eventually producing a thin line in the result. 

    In other words, we check if the gradient calculated in the previous step is considered the maximum among the neighboring points lying in both the positive and negative directions of the gradient. If the gradient was the maximum, it is considered to be part of the edge, and vice versa.

    The tracking process above is controlled by two thresholds, t1 and t2, such that t1>t2, referred to as hysteresis thresholding. Tracking begins at a point on the ridge higher than t1, and then continues in both of the directions out of that point until the height of the ridge becomes less than t2. 

    So, basically, what happens here is that we select all the edge points that are above the upper threshold t1, and then investigate if there are neighbors of these points which are considered below the upper threshold t1 and above the lower threshold t2. In this case, such neighbors would be part of the edge.

    Thus, the width of the Gaussian kernel used for smoothing the input image, and the t1 (upper) and t2 (lower) thresholds used by the tracker, are the parameters that determine the effect of the canny edge detector.

    Python Implementation

    In this section, I will describe two ways in which we can implement the Canny edge detector. One way uses the scikit-image library, and the other uses the OpenCV library.

    Canny Edge Detector Using scikit-image

    If you don't have scikit-image already installed on your machine, go ahead and install it by following the instructions shown on the installing scikit-image page.

    As I'm using an Ubuntu machine, I simply had to run the following command in my Terminal to get the library up and running:

    sudo apt-get install python-skimage

    The scikit-image library has a canny() function which we can use to apply the Canny edge detector on our image. Notice that the function is part of the feature module.

    Before moving forward, let's use a toy image to experiment with. You can use any image though. I'm going to use the boat.png image shown below (click on the link to download the image):

    Without further ado, let's see how we can detect the edges in the above image (i.e. boat) using the Canny edge detector. Remember that our image needs to be grayscale. Since our image is already grayscale, we don't need to do anything at this point, such as converting the image from color to grayscale. The script for the Canny edge detector looks as follows:

    from skimage import io from skimage import feature im = io.imread('boat.png') edges = feature.canny(im) io.imshow(edges) io.show()

    So, as you can see, we first read our image, boat.png. After that, we apply the canny() function on the image (I didn't pass any custom parameters, except our image, and left it at the function's defaults). Finally, we display our result that shows the detected edges. The result of the above script looks as follows:

    You can play around with the parameters to get different results on how edges are detected. But the result looks nice with those detected edges, doesn't it?!

    Canny Edge Detector Using OpenCV

    In this section, we are going to see how we can use OpenCV to apply the Canny edge detector on our boat image. If you don't have OpenCV installed yet, go ahead and install it. You can check the following articles on how you can install OpenCV on your machine. I have included different articles for different operating systems:

    As with the scikit-image library, OpenCV also has a function called canny() to apply the Canny edge detector algorithm on the image. The following script shows how we can use OpenCV to find the edges in our image:

    import cv2 import matplotlib.pyplot as plt im = cv2.imread('boat.png') edges = cv2.Canny(im,25,255,L2gradient=False) plt.imshow(edges,cmap='gray') plt.show()

    Notice that I have passed the following as arguments to the Canny() function:

    • im: image name
    • lower threshold: 25
    • upper threshold: 255
    • L2gradient=False: this means that the L1-norm is used. If set to True, the L2-norm will be used.

    The matplotlib library has then been used to plot the results. To learn more about this library, check my tutorial: Introducing Python's Matplotlib Library

    The result of the above script is as follows:

    Conclusion

    In this tutorial, we have learned about the Canny edge detector and seen how the scikit-image and OpenCV libraries enable us to easily implement this detector with a few lines of code.

    6 days 22 hours ago

Pages